2025.02.03 16:47:40.795500 [ 349 ] {} Context: Server was built with undefined behavior sanitizer. It will work slowly. 2025.02.03 16:47:40.817494 [ 349 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2025.02.03 16:47:40.818028 [ 349 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2025.02.03 16:47:41.377087 [ 349 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2025.02.03 16:47:41.377436 [ 349 ] {} KeeperLogStore: Removing all changelogs 2025.02.03 16:47:41.378156 [ 349 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2025.02.03 16:47:41.381261 [ 349 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2025.02.03 16:47:41.381283 [ 349 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2025.02.03 16:47:41.431924 [ 935 ] {} RaftInstance: Election timeout, initiate leader election 2025.02.03 16:47:41.455189 [ 349 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2025.02.03 16:47:41.455218 [ 349 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2025.02.03 16:47:41.533870 [ 349 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2025.02.03 16:47:42.638767 [ 353 ] {d4ad1ae7-e5fe-405a-8599-aacaebd2b52b} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2025.02.03 16:47:52.970401 [ 353 ] {0258040f-1d5f-4fd1-b344-90fa44c101f4} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2025.02.03 16:48:27.526995 [ 1733 ] {f28680e5-94c8-422e-a7d1-88b31d893938} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.071299981 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:40500) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:27.546747 [ 1733 ] {f28680e5-94c8-422e-a7d1-88b31d893938} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.071299981 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:27.631729 [ 1779 ] {f67f67a5-c342-4511-b7ca-a0fc2974ee7d} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42654). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42654) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:27.651408 [ 1779 ] {f67f67a5-c342-4511-b7ca-a0fc2974ee7d} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42654). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:27.651477 [ 1779 ] {f67f67a5-c342-4511-b7ca-a0fc2974ee7d} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42654). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:27.651641 [ 1779 ] {f67f67a5-c342-4511-b7ca-a0fc2974ee7d} TCPHandler: Client has gone away. 2025.02.03 16:48:28.811136 [ 1733 ] {26c64a7b-8fda-4b99-a4be-ed62fd811a19} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.082617982 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:40512) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:28.827360 [ 1733 ] {26c64a7b-8fda-4b99-a4be-ed62fd811a19} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.082617982 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:28.862358 [ 353 ] {678c59f8-0ce0-4def-bdf5-6612567a8163} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42646). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42646) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:28.866989 [ 353 ] {678c59f8-0ce0-4def-bdf5-6612567a8163} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42646). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:28.867060 [ 353 ] {678c59f8-0ce0-4def-bdf5-6612567a8163} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42646). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:28.867200 [ 353 ] {678c59f8-0ce0-4def-bdf5-6612567a8163} TCPHandler: Client has gone away. 2025.02.03 16:48:28.922349 [ 1779 ] {eaa7dbdc-b3ac-4dcf-b531-619fabf482d9} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55146). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:55146) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:28.931434 [ 1779 ] {eaa7dbdc-b3ac-4dcf-b531-619fabf482d9} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55146). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:28.931531 [ 1779 ] {eaa7dbdc-b3ac-4dcf-b531-619fabf482d9} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55146). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:28.931666 [ 1779 ] {eaa7dbdc-b3ac-4dcf-b531-619fabf482d9} TCPHandler: Client has gone away. 2025.02.03 16:48:30.155283 [ 1681 ] {cfe0c482-69a9-4455-a4af-d90ea2d23ca4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.108624074 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:40520) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:30.173398 [ 1681 ] {cfe0c482-69a9-4455-a4af-d90ea2d23ca4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.108624074 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:30.179465 [ 353 ] {a3ce58bc-edd5-48e6-b87e-2a10c0063dd4} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42660). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42660) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:30.184274 [ 353 ] {a3ce58bc-edd5-48e6-b87e-2a10c0063dd4} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42660). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:30.184380 [ 353 ] {a3ce58bc-edd5-48e6-b87e-2a10c0063dd4} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42660). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:30.184509 [ 353 ] {a3ce58bc-edd5-48e6-b87e-2a10c0063dd4} TCPHandler: Client has gone away. 2025.02.03 16:48:30.220443 [ 1732 ] {cf5b8c8f-437f-4fb1-bf17-5bac00a08f81} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:41988). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:41988) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:30.227954 [ 1732 ] {cf5b8c8f-437f-4fb1-bf17-5bac00a08f81} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:41988). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:30.228027 [ 1732 ] {cf5b8c8f-437f-4fb1-bf17-5bac00a08f81} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:41988). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:30.228221 [ 1732 ] {cf5b8c8f-437f-4fb1-bf17-5bac00a08f81} TCPHandler: Client has gone away. 2025.02.03 16:48:30.268354 [ 1733 ] {cb3822e4-4002-40cd-b0a3-0ad7e1da88e7} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55158). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:55158) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:30.273516 [ 1733 ] {cb3822e4-4002-40cd-b0a3-0ad7e1da88e7} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55158). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:30.273586 [ 1733 ] {cb3822e4-4002-40cd-b0a3-0ad7e1da88e7} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55158). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:30.273673 [ 1733 ] {cb3822e4-4002-40cd-b0a3-0ad7e1da88e7} TCPHandler: Client has gone away. 2025.02.03 16:48:31.509066 [ 1733 ] {3c0e51c2-e05a-4fcf-bcae-38c2e85b6c56} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.103933081 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:40526) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.521767 [ 1733 ] {3c0e51c2-e05a-4fcf-bcae-38c2e85b6c56} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.103933081 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.522328 [ 1779 ] {3b961053-5a15-46dc-96c3-2b82bfbc1c6d} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:41992). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:41992) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.531747 [ 1779 ] {3b961053-5a15-46dc-96c3-2b82bfbc1c6d} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:41992). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:31.532015 [ 1779 ] {3b961053-5a15-46dc-96c3-2b82bfbc1c6d} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:41992). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.532247 [ 1779 ] {3b961053-5a15-46dc-96c3-2b82bfbc1c6d} TCPHandler: Client has gone away. 2025.02.03 16:48:31.599301 [ 353 ] {5b085b62-c4d6-4d15-86f1-09e3f05cf331} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55162). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:55162) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./contrib/llvm-project/libcxx/include/__mutex_base:253: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ac7aa 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.599420 [ 1681 ] {279474ff-55d3-43f8-8734-c3d6ce439a1f} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38580). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:38580) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./contrib/llvm-project/libcxx/include/__mutex_base:253: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ac7aa 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.606459 [ 353 ] {5b085b62-c4d6-4d15-86f1-09e3f05cf331} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55162). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:31.606522 [ 353 ] {5b085b62-c4d6-4d15-86f1-09e3f05cf331} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55162). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./contrib/llvm-project/libcxx/include/__mutex_base:253: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ac7aa 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.606610 [ 353 ] {5b085b62-c4d6-4d15-86f1-09e3f05cf331} TCPHandler: Client has gone away. 2025.02.03 16:48:31.608804 [ 1732 ] {0a162ede-53fa-4f59-8b07-7b55aabb0d8a} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42676). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42676) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.610705 [ 1681 ] {279474ff-55d3-43f8-8734-c3d6ce439a1f} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38580). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:31.610781 [ 1681 ] {279474ff-55d3-43f8-8734-c3d6ce439a1f} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38580). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./contrib/llvm-project/libcxx/include/__mutex_base:253: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ac7aa 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.610864 [ 1681 ] {279474ff-55d3-43f8-8734-c3d6ce439a1f} TCPHandler: Client has gone away. 2025.02.03 16:48:31.620386 [ 1732 ] {0a162ede-53fa-4f59-8b07-7b55aabb0d8a} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42676). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:31.620601 [ 1732 ] {0a162ede-53fa-4f59-8b07-7b55aabb0d8a} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42676). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:31.620800 [ 1732 ] {0a162ede-53fa-4f59-8b07-7b55aabb0d8a} TCPHandler: Client has gone away. 2025.02.03 16:48:32.797118 [ 1732 ] {c017cff6-3660-411b-8e1d-bcf665c2b09e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.0537124 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:40546) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.810005 [ 1732 ] {c017cff6-3660-411b-8e1d-bcf665c2b09e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.0537124 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.866728 [ 1934 ] {13fc1a8c-3b30-42f0-9227-13ee442bbaf4} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38582). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:38582) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.871634 [ 1934 ] {13fc1a8c-3b30-42f0-9227-13ee442bbaf4} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38582). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:32.871758 [ 1934 ] {13fc1a8c-3b30-42f0-9227-13ee442bbaf4} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38582). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.871869 [ 1934 ] {13fc1a8c-3b30-42f0-9227-13ee442bbaf4} TCPHandler: Client has gone away. 2025.02.03 16:48:32.872867 [ 1779 ] {3a171cb3-a53a-40af-8205-bbe09bf2bf55} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55168). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:55168) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.877701 [ 1779 ] {3a171cb3-a53a-40af-8205-bbe09bf2bf55} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55168). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:32.877752 [ 1779 ] {3a171cb3-a53a-40af-8205-bbe09bf2bf55} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55168). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.877844 [ 1779 ] {3a171cb3-a53a-40af-8205-bbe09bf2bf55} TCPHandler: Client has gone away. 2025.02.03 16:48:32.888427 [ 1733 ] {a3b4d47d-a3f1-4155-b267-a4ab555cdb44} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:42490). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42490) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.895128 [ 1733 ] {a3b4d47d-a3f1-4155-b267-a4ab555cdb44} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:42490). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:32.895202 [ 1733 ] {a3b4d47d-a3f1-4155-b267-a4ab555cdb44} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:42490). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.895308 [ 1733 ] {a3b4d47d-a3f1-4155-b267-a4ab555cdb44} TCPHandler: Client has gone away. 2025.02.03 16:48:32.895992 [ 1681 ] {df9d3020-7d58-4b72-a4b8-5024c2ad99ea} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42000). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42000) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.896394 [ 353 ] {e5cc5ea3-692f-4785-86ac-6f20530e8e1d} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42680). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42680) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.901546 [ 1681 ] {df9d3020-7d58-4b72-a4b8-5024c2ad99ea} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:32.901589 [ 1681 ] {df9d3020-7d58-4b72-a4b8-5024c2ad99ea} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.901761 [ 1681 ] {df9d3020-7d58-4b72-a4b8-5024c2ad99ea} TCPHandler: Client has gone away. 2025.02.03 16:48:32.903055 [ 353 ] {e5cc5ea3-692f-4785-86ac-6f20530e8e1d} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42680). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:32.903120 [ 353 ] {e5cc5ea3-692f-4785-86ac-6f20530e8e1d} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42680). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:32.903203 [ 353 ] {e5cc5ea3-692f-4785-86ac-6f20530e8e1d} TCPHandler: Client has gone away. 2025.02.03 16:48:34.129054 [ 353 ] {498880d3-a3b6-4ea5-a7e3-b2b247082ef8} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.087952336 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:40562) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.140245 [ 353 ] {498880d3-a3b6-4ea5-a7e3-b2b247082ef8} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.087952336 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.188415 [ 1779 ] {ab27399c-9b41-434e-ae08-9cbec85a96d3} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:42498). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42498) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.190260 [ 1733 ] {5cbe23d0-63b1-4290-b324-ee1cf01fa032} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42010). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42010) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.191174 [ 1681 ] {08c25db6-d25a-45b6-a99c-3bda9b91cbfb} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42700). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42700) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.193054 [ 1779 ] {ab27399c-9b41-434e-ae08-9cbec85a96d3} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:42498). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:34.193111 [ 1779 ] {ab27399c-9b41-434e-ae08-9cbec85a96d3} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:42498). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.193212 [ 1779 ] {ab27399c-9b41-434e-ae08-9cbec85a96d3} TCPHandler: Client has gone away. 2025.02.03 16:48:34.196434 [ 1681 ] {08c25db6-d25a-45b6-a99c-3bda9b91cbfb} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42700). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:34.196488 [ 1681 ] {08c25db6-d25a-45b6-a99c-3bda9b91cbfb} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42700). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.196582 [ 1681 ] {08c25db6-d25a-45b6-a99c-3bda9b91cbfb} TCPHandler: Client has gone away. 2025.02.03 16:48:34.197110 [ 1733 ] {5cbe23d0-63b1-4290-b324-ee1cf01fa032} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42010). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:34.197187 [ 1733 ] {5cbe23d0-63b1-4290-b324-ee1cf01fa032} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42010). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.197285 [ 1733 ] {5cbe23d0-63b1-4290-b324-ee1cf01fa032} TCPHandler: Client has gone away. 2025.02.03 16:48:34.229009 [ 1934 ] {c4911d97-a4b2-4fc1-af3b-42b451601a5c} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38592). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:38592) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.230796 [ 1950 ] {a7d80d75-cd16-4db2-8c66-41e56f82d57b} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:57560). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:57560) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.232268 [ 1732 ] {2c7d0fbd-34d4-401d-8357-55b334b5ae0c} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55182). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:55182) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.236630 [ 1934 ] {c4911d97-a4b2-4fc1-af3b-42b451601a5c} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38592). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:34.236721 [ 1934 ] {c4911d97-a4b2-4fc1-af3b-42b451601a5c} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38592). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.236865 [ 1934 ] {c4911d97-a4b2-4fc1-af3b-42b451601a5c} TCPHandler: Client has gone away. 2025.02.03 16:48:34.239492 [ 1950 ] {a7d80d75-cd16-4db2-8c66-41e56f82d57b} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:57560). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:34.239585 [ 1950 ] {a7d80d75-cd16-4db2-8c66-41e56f82d57b} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:57560). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.239776 [ 1950 ] {a7d80d75-cd16-4db2-8c66-41e56f82d57b} TCPHandler: Client has gone away. 2025.02.03 16:48:34.242523 [ 1732 ] {2c7d0fbd-34d4-401d-8357-55b334b5ae0c} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55182). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:34.242662 [ 1732 ] {2c7d0fbd-34d4-401d-8357-55b334b5ae0c} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55182). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:34.242865 [ 1732 ] {2c7d0fbd-34d4-401d-8357-55b334b5ae0c} TCPHandler: Client has gone away. 2025.02.03 16:48:35.421583 [ 1681 ] {357f9f49-0e7e-4656-8fe8-1230e9606b73} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.097642993 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:40592) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.431625 [ 1681 ] {357f9f49-0e7e-4656-8fe8-1230e9606b73} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.097642993 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.523055 [ 353 ] {54930db0-6497-4e94-b577-3e0f4aee220a} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:57572). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:57572) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.523995 [ 1953 ] {a276e56b-9685-4727-a843-69861502b4a7} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:58384). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:58384) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.527568 [ 1779 ] {4a6c6d3c-fba1-4162-a141-2ff30ff3986b} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38602). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:38602) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.527657 [ 353 ] {54930db0-6497-4e94-b577-3e0f4aee220a} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:57572). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:35.527756 [ 353 ] {54930db0-6497-4e94-b577-3e0f4aee220a} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:57572). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.527852 [ 353 ] {54930db0-6497-4e94-b577-3e0f4aee220a} TCPHandler: Client has gone away. 2025.02.03 16:48:35.528430 [ 1732 ] {29af67c3-c0d4-4c15-9d4c-ac4ae19b7042} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42708). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42708) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.528448 [ 1950 ] {f7774417-85d2-4d5b-85eb-63f2d2063c81} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:42510). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42510) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.528732 [ 1953 ] {a276e56b-9685-4727-a843-69861502b4a7} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:58384). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:35.528789 [ 1953 ] {a276e56b-9685-4727-a843-69861502b4a7} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:58384). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.528876 [ 1953 ] {a276e56b-9685-4727-a843-69861502b4a7} TCPHandler: Client has gone away. 2025.02.03 16:48:35.529574 [ 1733 ] {71f3f942-8acd-4d4c-a994-81fe4cf788c7} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55192). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:55192) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.530168 [ 1934 ] {c8e474be-4c7d-497b-8d3e-7225083c005b} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42018). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42018) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.532413 [ 1779 ] {4a6c6d3c-fba1-4162-a141-2ff30ff3986b} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38602). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:35.532536 [ 1779 ] {4a6c6d3c-fba1-4162-a141-2ff30ff3986b} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:38602). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.532634 [ 1779 ] {4a6c6d3c-fba1-4162-a141-2ff30ff3986b} TCPHandler: Client has gone away. 2025.02.03 16:48:35.533206 [ 1732 ] {29af67c3-c0d4-4c15-9d4c-ac4ae19b7042} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42708). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:35.533356 [ 1732 ] {29af67c3-c0d4-4c15-9d4c-ac4ae19b7042} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42708). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.533444 [ 1732 ] {29af67c3-c0d4-4c15-9d4c-ac4ae19b7042} TCPHandler: Client has gone away. 2025.02.03 16:48:35.533927 [ 1950 ] {f7774417-85d2-4d5b-85eb-63f2d2063c81} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:42510). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:35.533979 [ 1950 ] {f7774417-85d2-4d5b-85eb-63f2d2063c81} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:42510). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.534085 [ 1950 ] {f7774417-85d2-4d5b-85eb-63f2d2063c81} TCPHandler: Client has gone away. 2025.02.03 16:48:35.535059 [ 1733 ] {71f3f942-8acd-4d4c-a994-81fe4cf788c7} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55192). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:35.535113 [ 1733 ] {71f3f942-8acd-4d4c-a994-81fe4cf788c7} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55192). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.535205 [ 1733 ] {71f3f942-8acd-4d4c-a994-81fe4cf788c7} TCPHandler: Client has gone away. 2025.02.03 16:48:35.535402 [ 1934 ] {c8e474be-4c7d-497b-8d3e-7225083c005b} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42018). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:35.535472 [ 1934 ] {c8e474be-4c7d-497b-8d3e-7225083c005b} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42018). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:35.535559 [ 1934 ] {c8e474be-4c7d-497b-8d3e-7225083c005b} TCPHandler: Client has gone away. 2025.02.03 16:48:36.712260 [ 1681 ] {4c2b8c0c-2ce5-41e3-a300-71e20e20cff5} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.061511863 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:40598) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.722990 [ 1681 ] {4c2b8c0c-2ce5-41e3-a300-71e20e20cff5} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.061511863 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.801245 [ 353 ] {8fbbf9cd-26de-46b9-b86e-a3903d58e396} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:53514). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:53514) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.801605 [ 1779 ] {2c63dd39-9941-4db4-a798-5b5f7cbb5271} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:52616). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:52616) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.802829 [ 1733 ] {f3c9ced6-fdf0-49ea-ad34-a1057a9fa1b0} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:36250). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:36250) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.807059 [ 353 ] {8fbbf9cd-26de-46b9-b86e-a3903d58e396} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:53514). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:36.807123 [ 353 ] {8fbbf9cd-26de-46b9-b86e-a3903d58e396} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:53514). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.807152 [ 1779 ] {2c63dd39-9941-4db4-a798-5b5f7cbb5271} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:52616). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:36.807230 [ 1779 ] {2c63dd39-9941-4db4-a798-5b5f7cbb5271} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:52616). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.807335 [ 1779 ] {2c63dd39-9941-4db4-a798-5b5f7cbb5271} TCPHandler: Client has gone away. 2025.02.03 16:48:36.807235 [ 353 ] {8fbbf9cd-26de-46b9-b86e-a3903d58e396} TCPHandler: Client has gone away. 2025.02.03 16:48:36.808369 [ 1733 ] {f3c9ced6-fdf0-49ea-ad34-a1057a9fa1b0} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:36250). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:36.808427 [ 1733 ] {f3c9ced6-fdf0-49ea-ad34-a1057a9fa1b0} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:36250). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.808525 [ 1733 ] {f3c9ced6-fdf0-49ea-ad34-a1057a9fa1b0} TCPHandler: Client has gone away. 2025.02.03 16:48:36.808590 [ 1732 ] {c5d28bad-01fc-4e47-a3d8-a380092d7a38} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:43978). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:43978) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.813071 [ 1732 ] {c5d28bad-01fc-4e47-a3d8-a380092d7a38} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:43978). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:36.813135 [ 1732 ] {c5d28bad-01fc-4e47-a3d8-a380092d7a38} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:43978). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.813304 [ 1732 ] {c5d28bad-01fc-4e47-a3d8-a380092d7a38} TCPHandler: Client has gone away. 2025.02.03 16:48:36.814423 [ 1950 ] {d91ebdf6-060e-4c88-af43-767123ce4b19} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:53688). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:53688) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.818652 [ 1934 ] {1d7aede4-40c0-44fc-a8d9-54a2ca1ea17f} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:32870). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:32870) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.819695 [ 1952 ] {cf18679a-c830-4be8-b53d-2d18789dc167} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:51720). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:51720) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.819758 [ 1953 ] {8657ec00-86b4-43e5-b000-de3b6427ed69} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:56974). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:56974) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.820390 [ 1950 ] {d91ebdf6-060e-4c88-af43-767123ce4b19} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:53688). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:36.820441 [ 1950 ] {d91ebdf6-060e-4c88-af43-767123ce4b19} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:53688). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.820536 [ 1950 ] {d91ebdf6-060e-4c88-af43-767123ce4b19} TCPHandler: Client has gone away. 2025.02.03 16:48:36.823670 [ 1934 ] {1d7aede4-40c0-44fc-a8d9-54a2ca1ea17f} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:32870). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:36.823742 [ 1934 ] {1d7aede4-40c0-44fc-a8d9-54a2ca1ea17f} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:32870). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.823827 [ 1934 ] {1d7aede4-40c0-44fc-a8d9-54a2ca1ea17f} TCPHandler: Client has gone away. 2025.02.03 16:48:36.824661 [ 1952 ] {cf18679a-c830-4be8-b53d-2d18789dc167} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:51720). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:36.824749 [ 1952 ] {cf18679a-c830-4be8-b53d-2d18789dc167} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:51720). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.824832 [ 1953 ] {8657ec00-86b4-43e5-b000-de3b6427ed69} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:56974). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:36.824837 [ 1952 ] {cf18679a-c830-4be8-b53d-2d18789dc167} TCPHandler: Client has gone away. 2025.02.03 16:48:36.824884 [ 1953 ] {8657ec00-86b4-43e5-b000-de3b6427ed69} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:56974). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:36.824993 [ 1953 ] {8657ec00-86b4-43e5-b000-de3b6427ed69} TCPHandler: Client has gone away. 2025.02.03 16:48:37.974851 [ 1681 ] {b32dae71-8889-40d7-9ec0-256c3711456d} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.069295046 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:47020) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:37.985716 [ 1681 ] {b32dae71-8889-40d7-9ec0-256c3711456d} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.069295046 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000003056ddef 5. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000003056d5c3 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x00000000346f2814 7. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000346f2a79 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000346f2625 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003470af6c 10. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 12. ? @ 0x00007fb25a769ac3 13. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.060443 [ 353 ] {79176575-b276-4b99-994c-3f472b8ecfdc} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:51728). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:51728) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.062836 [ 1732 ] {d3acbcaa-b73c-479d-9330-1695bd49da52} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:56976). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:56976) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.065628 [ 353 ] {79176575-b276-4b99-994c-3f472b8ecfdc} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:51728). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:38.065728 [ 353 ] {79176575-b276-4b99-994c-3f472b8ecfdc} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:51728). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.065903 [ 353 ] {79176575-b276-4b99-994c-3f472b8ecfdc} TCPHandler: Client has gone away. 2025.02.03 16:48:38.066808 [ 1732 ] {d3acbcaa-b73c-479d-9330-1695bd49da52} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:56976). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:38.066860 [ 1732 ] {d3acbcaa-b73c-479d-9330-1695bd49da52} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:56976). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.066949 [ 1732 ] {d3acbcaa-b73c-479d-9330-1695bd49da52} TCPHandler: Client has gone away. 2025.02.03 16:48:38.067483 [ 1733 ] {6e59191a-cbb4-411c-975e-50b38694c186} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:32884). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:32884) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.069164 [ 1779 ] {88682d26-5754-4857-9fbb-eb1f5acc33bd} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:53702). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:53702) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.069852 [ 2003 ] {d728dccb-85a8-40e9-b251-9dd04dfefb68} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:43980). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:43980) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.069922 [ 1950 ] {af52f83c-adae-4e7f-adcb-03883e923816} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:36264). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:36264) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.070604 [ 1952 ] {6123a03f-2cbb-4117-aff5-85db3c44abaf} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:52632). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:52632) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.071671 [ 1934 ] {546e81ec-6d6d-4f0b-9887-a45440b6918d} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:43480). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:43480) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.072569 [ 1733 ] {6e59191a-cbb4-411c-975e-50b38694c186} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:32884). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:38.072617 [ 1733 ] {6e59191a-cbb4-411c-975e-50b38694c186} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:32884). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.072732 [ 1733 ] {6e59191a-cbb4-411c-975e-50b38694c186} TCPHandler: Client has gone away. 2025.02.03 16:48:38.075369 [ 1779 ] {88682d26-5754-4857-9fbb-eb1f5acc33bd} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:53702). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:38.075441 [ 1779 ] {88682d26-5754-4857-9fbb-eb1f5acc33bd} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:53702). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.075545 [ 1779 ] {88682d26-5754-4857-9fbb-eb1f5acc33bd} TCPHandler: Client has gone away. 2025.02.03 16:48:38.076881 [ 1934 ] {546e81ec-6d6d-4f0b-9887-a45440b6918d} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:43480). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:38.076959 [ 1934 ] {546e81ec-6d6d-4f0b-9887-a45440b6918d} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:43480). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.077065 [ 1934 ] {546e81ec-6d6d-4f0b-9887-a45440b6918d} TCPHandler: Client has gone away. 2025.02.03 16:48:38.078494 [ 1953 ] {46455bf8-2f06-4266-80ee-86d96e0489ac} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:53520). (NETWORK_ERROR) (version 24.8.11.51000.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:53520) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.078902 [ 2003 ] {d728dccb-85a8-40e9-b251-9dd04dfefb68} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:43980). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:38.078960 [ 2003 ] {d728dccb-85a8-40e9-b251-9dd04dfefb68} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:43980). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.079055 [ 2003 ] {d728dccb-85a8-40e9-b251-9dd04dfefb68} TCPHandler: Client has gone away. 2025.02.03 16:48:38.079561 [ 1950 ] {af52f83c-adae-4e7f-adcb-03883e923816} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:36264). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:38.079612 [ 1950 ] {af52f83c-adae-4e7f-adcb-03883e923816} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:36264). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.079664 [ 1952 ] {6123a03f-2cbb-4117-aff5-85db3c44abaf} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:52632). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:38.079758 [ 1950 ] {af52f83c-adae-4e7f-adcb-03883e923816} TCPHandler: Client has gone away. 2025.02.03 16:48:38.079771 [ 1952 ] {6123a03f-2cbb-4117-aff5-85db3c44abaf} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:52632). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2476: DB::TCPHandler::sendProgress() @ 0x00000000345ad6ef 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345ace85 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.079872 [ 1952 ] {6123a03f-2cbb-4117-aff5-85db3c44abaf} TCPHandler: Client has gone away. 2025.02.03 16:48:38.082060 [ 1953 ] {46455bf8-2f06-4266-80ee-86d96e0489ac} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:53520). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bfff 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000345aeadc 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000034594754 8. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 13. ? @ 0x00007fb25a769ac3 14. ? @ 0x00007fb25a7fb850 (version 24.8.11.51000.altinitytest (altinity build)) 2025.02.03 16:48:38.082106 [ 1953 ] {46455bf8-2f06-4266-80ee-86d96e0489ac} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:53520). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025a483e7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025a4bb7f 5. DB::WriteBuffer::next() @ 0x0000000018eef8dc 6. ./build_docker/./src/Server/TCPHandler.cpp:2406: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000345b23e3 7. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000345acd52 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000345941ff 9. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:48:38.082190 [ 1953 ] {46455bf8-2f06-4266-80ee-86d96e0489ac} TCPHandler: Client has gone away. 2025.02.03 16:49:48.334233 [ 1952 ] {1405c04c-7c5b-4528-9d5b-d7dadcae1469} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:53522) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:263: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350e66d2 7. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003507055c 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350a7f21 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003199273e 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000319921dc 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:537: DB::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030ebf716 12. ./contrib/llvm-project/libcxx/include/vector:553: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000030eb001f 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3910: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000030eae3cd 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030ed7580 15. ./contrib/llvm-project/libcxx/include/vector:553: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000030eafa5f 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3910: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000030eae3cd 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030efb46c 18. ./contrib/llvm-project/libcxx/include/vector:961: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x0000000030ea1e28 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000030e9f139 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000030e9e9ca 21. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x00000000317e10b1 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000003199132c 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000003198dfdc 24. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x00000000319936b7 25. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000031992f05 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000318cd39c 27. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000031eedb07 28. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031ee7ed3 29. ./build_docker/./src/Server/TCPHandler.cpp:569: DB::TCPHandler::runImpl() @ 0x0000000034593e28 30. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 31. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 2025.02.03 16:49:48.334583 [ 1952 ] {1405c04c-7c5b-4528-9d5b-d7dadcae1469} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:263: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350e66d2 7. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003507055c 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350a7f21 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003199273e 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000319921dc 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:537: DB::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030ebf716 12. ./contrib/llvm-project/libcxx/include/vector:553: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000030eb001f 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3910: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000030eae3cd 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030ed7580 15. ./contrib/llvm-project/libcxx/include/vector:553: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000030eafa5f 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3910: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000030eae3cd 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030efb46c 18. ./contrib/llvm-project/libcxx/include/vector:961: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x0000000030ea1e28 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000030e9f139 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000030e9e9ca 21. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x00000000317e10b1 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000003199132c 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000003198dfdc 24. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x00000000319936b7 25. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000031992f05 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000318cd39c 27. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000031eedb07 28. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031ee7ed3 29. ./build_docker/./src/Server/TCPHandler.cpp:569: DB::TCPHandler::runImpl() @ 0x0000000034593e28 30. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 31. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 2025.02.03 16:49:48.380507 [ 1952 ] {ed849b93-151b-4740-9338-3253bcea265b} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:53522) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:263: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350e66d2 7. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003507055c 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350a7f21 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003199273e 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000319921dc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000031eee926 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031ee7ed3 13. ./build_docker/./src/Server/TCPHandler.cpp:569: DB::TCPHandler::runImpl() @ 0x0000000034593e28 14. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 18. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 19. ? @ 0x00007fb25a769ac3 20. ? @ 0x00007fb25a7fb850 2025.02.03 16:49:48.380753 [ 1952 ] {ed849b93-151b-4740-9338-3253bcea265b} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:263: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350e66d2 7. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003507055c 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350a7f21 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003199273e 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000319921dc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000031eee926 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031ee7ed3 13. ./build_docker/./src/Server/TCPHandler.cpp:569: DB::TCPHandler::runImpl() @ 0x0000000034593e28 14. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 18. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000037fa0a4f 19. ? @ 0x00007fb25a769ac3 20. ? @ 0x00007fb25a7fb850 2025.02.03 16:49:48.528191 [ 1952 ] {3fe57c86-293b-445b-87a5-63b3a574a9f8} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.78 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:53522) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:49:48.529390 [ 1952 ] {3fe57c86-293b-445b-87a5-63b3a574a9f8} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.78 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:49:48.618695 [ 1952 ] {3940f95c-757f-4f3c-8e45-fadb25e226d5} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:53522) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:263: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350e66d2 7. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003507055c 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350a7f21 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003199273e 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000319921dc 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:537: DB::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030ebf716 12. ./contrib/llvm-project/libcxx/include/vector:553: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000030eb001f 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3910: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000030eae3cd 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030ed7580 15. ./contrib/llvm-project/libcxx/include/vector:553: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000030eafa5f 16. ./contrib/llvm-project/libcxx/include/vector:432: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x0000000030ea223d 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000030e9f139 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000030e9e9ca 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x00000000317e10b1 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000003199132c 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000003198dfdc 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x00000000319936b7 23. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000031992f05 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000318cd39c 25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000031eedb07 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031ee7ed3 27. ./build_docker/./src/Server/TCPHandler.cpp:569: DB::TCPHandler::runImpl() @ 0x0000000034593e28 28. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 2025.02.03 16:49:48.618950 [ 1952 ] {3940f95c-757f-4f3c-8e45-fadb25e226d5} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:263: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350e66d2 7. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003507055c 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000350a7f21 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003199273e 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000319921dc 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:537: DB::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030ebf716 12. ./contrib/llvm-project/libcxx/include/vector:553: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000030eb001f 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3910: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000030eae3cd 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000030ed7580 15. ./contrib/llvm-project/libcxx/include/vector:553: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000030eafa5f 16. ./contrib/llvm-project/libcxx/include/vector:432: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x0000000030ea223d 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000030e9f139 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000030e9e9ca 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x00000000317e10b1 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000003199132c 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000003198dfdc 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x00000000319936b7 23. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000031992f05 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000318cd39c 25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000031eedb07 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031ee7ed3 27. ./build_docker/./src/Server/TCPHandler.cpp:569: DB::TCPHandler::runImpl() @ 0x0000000034593e28 28. ./build_docker/./src/Server/TCPHandler.cpp:2516: DB::TCPHandler::run() @ 0x00000000345cb136 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000380511f8 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038052332 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000037fa5a81 2025.02.03 16:49:48.736946 [ 1952 ] {f46a3a4b-e083-4556-bbf1-8c1060f5a303} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:53522) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:49:48.737204 [ 1952 ] {f46a3a4b-e083-4556-bbf1-8c1060f5a303} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x00000000305b198f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b173e 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:51:39.491545 [ 1950 ] {70bd0794-780b-469f-aea2-b21689ae3f25} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:39784) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x00000000305b1a2f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:35: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b1823 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:51:39.492356 [ 1950 ] {70bd0794-780b-469f-aea2-b21689ae3f25} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x00000000305b1a2f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:35: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b1823 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:51:39.819639 [ 1950 ] {462517d2-278d-475e-a220-c3e75c20d057} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.69 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:39784) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x00000000305b1a2f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:35: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b1823 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:51:39.820616 [ 1950 ] {462517d2-278d-475e-a220-c3e75c20d057} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.69 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x00000000305b1a2f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:35: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b1823 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:51:40.200578 [ 1950 ] {dc7fcd77-faa6-4239-9946-2ebb6e2c80fd} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.8.11.51000.altinitytest (altinity build)) (from [::1]:39784) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x00000000305b1a2f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:35: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b1823 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:51:40.201388 [ 1950 ] {dc7fcd77-faa6-4239-9946-2ebb6e2c80fd} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018ee598b 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000037f006c5 2. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025822c8b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018ebd136 4. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x00000000305b1a2f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:35: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000305b1823 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000003056c87e 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000034705c55 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000346f37e3 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000346f5069 10. ./build_docker/./src/Common/ThreadPool.cpp:781: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000025957a5c 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002595fb14 12. ./base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x00000000259551af 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000002595c3aa 14. ? @ 0x00007fb25a769ac3 15. ? @ 0x00007fb25a7fb850 2025.02.03 16:54:59.225597 [ 349 ] {} Application: Closed all listening sockets. Waiting for 1 outstanding connections. 2025.02.03 16:55:04.132187 [ 349 ] {} Application: Closed connections. But 1 remain. Tip: To increase wait time add to config: 60 2025.02.03 16:55:04.132317 [ 349 ] {} Application: Will shutdown forcefully.