2024.07.22 12:07:09.427219 [ 384 ] {} Context: Server was built with sanitizer. It will work slowly. 2024.07.22 12:07:09.441269 [ 384 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2024.07.22 12:07:09.441702 [ 384 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.07.22 12:07:09.809850 [ 384 ] {} CloudPlacementInfo: Placement info has not been loaded 2024.07.22 12:07:09.871220 [ 384 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2024.07.22 12:07:09.871408 [ 384 ] {} KeeperLogStore: Removing all changelogs 2024.07.22 12:07:09.871696 [ 384 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.07.22 12:07:09.872505 [ 384 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.07.22 12:07:09.872514 [ 384 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.07.22 12:07:09.922919 [ 963 ] {} RaftInstance: Election timeout, initiate leader election 2024.07.22 12:07:09.937236 [ 384 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2024.07.22 12:07:09.937254 [ 384 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2024.07.22 12:07:09.973401 [ 384 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.07.22 12:07:11.390617 [ 387 ] {7eb8f63f-bda2-411e-b133-03ac46447684} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.07.22 12:07:19.662448 [ 387 ] {44e95c65-b07d-4c0f-8387-9d3ea7d7f922} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.07.22 12:08:46.920688 [ 387 ] {269f54a2-11e4-442c-94fe-d242c76a18e2} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:35632) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033f0252c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e97d85 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ec927d 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fb3d3e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fb37dc 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2109: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030892677 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030834dfc 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030833401 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003085739d 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030834883 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030833401 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000308427ec 17. ./contrib/llvm-project/libcxx/include/vector:951: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003082086a 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000003081ce9d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x00000000308159e6 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000030fb29b6 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000030fafac3 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000030fb5318 23. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000030fb4528 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000030ed1c23 25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000316a7179 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000316a0667 27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033443e86 28. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 2024.07.22 12:08:46.920913 [ 387 ] {269f54a2-11e4-442c-94fe-d242c76a18e2} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033f0252c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e97d85 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ec927d 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fb3d3e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fb37dc 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2109: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030892677 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030834dfc 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030833401 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003085739d 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030834883 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030833401 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000308427ec 17. ./contrib/llvm-project/libcxx/include/vector:951: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003082086a 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000003081ce9d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x00000000308159e6 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000030fb29b6 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000030fafac3 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000030fb5318 23. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000030fb4528 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000030ed1c23 25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000316a7179 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000316a0667 27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033443e86 28. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 2024.07.22 12:08:46.941696 [ 387 ] {25dcf7b9-d192-4e39-a01b-16615b013c95} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:35632) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033f0252c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e97d85 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ec927d 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fb3d3e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fb37dc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000316a7fbe 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000316a0667 12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033443e86 13. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 17. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 18. ? @ 0x00007fb3e4339ac3 19. ? @ 0x00007fb3e43cb850 2024.07.22 12:08:46.941927 [ 387 ] {25dcf7b9-d192-4e39-a01b-16615b013c95} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033f0252c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e97d85 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ec927d 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fb3d3e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fb37dc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000316a7fbe 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000316a0667 12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033443e86 13. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 17. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 18. ? @ 0x00007fb3e4339ac3 19. ? @ 0x00007fb3e43cb850 2024.07.22 12:08:47.108240 [ 387 ] {ce5d8fe6-e792-4b13-879a-0526d7f37acd} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 15.82 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:35632) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:08:47.115378 [ 387 ] {ce5d8fe6-e792-4b13-879a-0526d7f37acd} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 15.82 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:08:47.150017 [ 387 ] {7c0da481-c37e-49b8-a732-d5b7feb686ff} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:35632) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033f0252c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e97d85 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ec927d 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fb3d3e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fb37dc 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2109: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030892677 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030834dfc 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030833401 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003085739d 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030834883 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030820ca1 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000003081ce9d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x00000000308159e6 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000030fb29b6 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000030fafac3 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000030fb5318 21. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000030fb4528 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000030ed1c23 23. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000316a7179 24. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000316a0667 25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033443e86 26. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 30. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 31. ? @ 0x00007fb3e4339ac3 2024.07.22 12:08:47.150262 [ 387 ] {7c0da481-c37e-49b8-a732-d5b7feb686ff} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033f0252c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e97d85 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ec927d 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fb3d3e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fb37dc 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2109: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030892677 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030834dfc 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030833401 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003085739d 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030834883 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030820ca1 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000003081ce9d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x00000000308159e6 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000030fb29b6 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000030fafac3 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000030fb5318 21. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000030fb4528 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000030ed1c23 23. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000316a7179 24. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000316a0667 25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033443e86 26. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 30. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 31. ? @ 0x00007fb3e4339ac3 2024.07.22 12:08:47.252699 [ 387 ] {0f33fa07-68ab-4609-b4c2-9c3be41cf3a8} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.38 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:35632) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:08:47.253899 [ 387 ] {0f33fa07-68ab-4609-b4c2-9c3be41cf3a8} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.38 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f4038b1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f403609 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:05.473232 [ 388 ] {9bb470f8-1f6b-4ca9-aae8-667f651c01ce} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 578.31 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:49028) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f403b52 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f4036e2 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:05.479230 [ 388 ] {9bb470f8-1f6b-4ca9-aae8-667f651c01ce} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 578.31 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f403b52 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f4036e2 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:05.635683 [ 388 ] {59ab7341-e005-4761-9903-497db2289e78} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 588.79 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:49028) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f403b52 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f4036e2 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:05.642647 [ 388 ] {59ab7341-e005-4761-9903-497db2289e78} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 588.79 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f403b52 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f4036e2 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:05.721226 [ 388 ] {fd66dc21-2023-4b78-9f4d-2fdc294f250a} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 582.58 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:49028) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f403b52 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f4036e2 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:05.724398 [ 388 ] {fd66dc21-2023-4b78-9f4d-2fdc294f250a} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 582.58 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f403b52 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f4036e2 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dcd5d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:08.315036 [ 2032 ] {e591c8b3-d180-411e-a231-f06f34c5c03b} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.002588067 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58166) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:08.335101 [ 2032 ] {e591c8b3-d180-411e-a231-f06f34c5c03b} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.002588067 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:08.338773 [ 1822 ] {416fea25-4e36-48c6-8882-eaa304bcd8ed} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.060569929 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47416) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:08.339848 [ 1683 ] {79584418-adb8-4642-be19-bd0a1376edf6} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.073063293 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:49058) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:08.341275 [ 1683 ] {79584418-adb8-4642-be19-bd0a1376edf6} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.073063293 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:08.381472 [ 2032 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:58166). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:08.392933 [ 1822 ] {416fea25-4e36-48c6-8882-eaa304bcd8ed} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.060569929 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:08.393248 [ 1822 ] {416fea25-4e36-48c6-8882-eaa304bcd8ed} TCPHandler: Client has gone away. 2024.07.22 12:10:09.599867 [ 1683 ] {81474ce8-0fc3-4132-a603-1a455065a361} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028549885 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:49062) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:09.602358 [ 1683 ] {81474ce8-0fc3-4132-a603-1a455065a361} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028549885 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:09.632058 [ 1822 ] {59c44b65-446b-45eb-a0b7-bf32aee3a6bb} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.055127699 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58180) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:09.686711 [ 1822 ] {59c44b65-446b-45eb-a0b7-bf32aee3a6bb} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:58180). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002483a9c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002483970f 5. DB::WriteBuffer::next() @ 0x000000001803868c 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000334647a3 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x00000000334447b9 8. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:09.686800 [ 1822 ] {59c44b65-446b-45eb-a0b7-bf32aee3a6bb} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.055127699 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:09.687018 [ 1822 ] {59c44b65-446b-45eb-a0b7-bf32aee3a6bb} TCPHandler: Client has gone away. 2024.07.22 12:10:10.920840 [ 1822 ] {77437539-460c-482d-9bc1-e56da3b82a94} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116995339 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58184) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:10.940154 [ 1822 ] {77437539-460c-482d-9bc1-e56da3b82a94} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116995339 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.079545 [ 2032 ] {4a0d47b5-3b24-449d-9180-a52d51fa9a00} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.505074479 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47426) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.110492 [ 1606 ] {ee05c703-33a8-4e28-888b-af0f41fcfc31} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.538313568 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35818) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.111415 [ 2032 ] {4a0d47b5-3b24-449d-9180-a52d51fa9a00} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.505074479 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.111760 [ 2032 ] {4a0d47b5-3b24-449d-9180-a52d51fa9a00} TCPHandler: Client has gone away. 2024.07.22 12:10:11.127600 [ 1606 ] {ee05c703-33a8-4e28-888b-af0f41fcfc31} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.538313568 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.127850 [ 1606 ] {ee05c703-33a8-4e28-888b-af0f41fcfc31} TCPHandler: Client has gone away. 2024.07.22 12:10:11.261330 [ 2771 ] {be6a8a8e-c083-4319-9ee4-d780f0b498da} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.461482229 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47440) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.284346 [ 2771 ] {be6a8a8e-c083-4319-9ee4-d780f0b498da} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.461482229 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.414713 [ 2772 ] {e0ff1b1b-51e3-41ff-b4dd-39905877be03} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.614671432 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35832) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.447959 [ 2772 ] {e0ff1b1b-51e3-41ff-b4dd-39905877be03} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.614671432 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.467130 [ 2033 ] {9b27ecf4-08ca-42fc-b76b-d89cf8255368} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.66762966 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34288) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.486271 [ 2033 ] {9b27ecf4-08ca-42fc-b76b-d89cf8255368} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.66762966 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.494708 [ 1683 ] {7a0e28e7-b9f5-4502-b712-0497b5d9cfa0} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.012021274 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:49064) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:11.495821 [ 2033 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:34288). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:11.495974 [ 2772 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:35832). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:11.496077 [ 2771 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:47440). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:11.496356 [ 1822 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:58184). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:11.496397 [ 1683 ] {7a0e28e7-b9f5-4502-b712-0497b5d9cfa0} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.012021274 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:12.748867 [ 2032 ] {a81492a7-daf7-48e7-ab29-87a91d04b7f5} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.066446095 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49666) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:12.815598 [ 2032 ] {a81492a7-daf7-48e7-ab29-87a91d04b7f5} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.066446095 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:12.828637 [ 1683 ] {842703fc-f2b5-4ad1-b120-3ad0f2314e43} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.156459831 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:42876) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:12.830835 [ 2032 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:49666). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:12.831801 [ 1683 ] {842703fc-f2b5-4ad1-b120-3ad0f2314e43} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.156459831 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.175802 [ 1683 ] {dd25073f-0821-43da-b67c-895616492251} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.098149281 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49678) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.196081 [ 1683 ] {dd25073f-0821-43da-b67c-895616492251} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.098149281 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.217380 [ 2773 ] {b5b64676-6e27-4e72-9e17-af59b353017c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.139945031 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39044) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.256947 [ 388 ] {85fe1c30-d73b-47f8-8abb-4c7c1349b35f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.192574145 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45972) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.260769 [ 2773 ] {b5b64676-6e27-4e72-9e17-af59b353017c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.139945031 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.277350 [ 388 ] {85fe1c30-d73b-47f8-8abb-4c7c1349b35f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.192574145 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.290131 [ 2775 ] {f511dfed-99de-4655-8987-b6780a9c5ae4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.2264523 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48786) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.316451 [ 2775 ] {f511dfed-99de-4655-8987-b6780a9c5ae4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.2264523 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.357172 [ 2010 ] {c1c0808c-d4e6-433e-930a-e7530875a3b3} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.298082017 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54044) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.389667 [ 2010 ] {c1c0808c-d4e6-433e-930a-e7530875a3b3} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.298082017 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.547954 [ 2033 ] {4e13f254-a43e-4089-812a-ab223f000978} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.876382029 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45962) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.568784 [ 2033 ] {4e13f254-a43e-4089-812a-ab223f000978} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.876382029 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.569621 [ 2033 ] {4e13f254-a43e-4089-812a-ab223f000978} TCPHandler: Client has gone away. 2024.07.22 12:10:14.710949 [ 2771 ] {c312f376-6aaf-4f68-9b5b-249c234cfdf3} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.039149958 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54042) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.715376 [ 1822 ] {ce9c833c-0e9d-4e85-934e-6c0f68c4b720} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.04416581 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39032) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.734086 [ 2771 ] {c312f376-6aaf-4f68-9b5b-249c234cfdf3} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.039149958 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.734450 [ 2771 ] {c312f376-6aaf-4f68-9b5b-249c234cfdf3} TCPHandler: Client has gone away. 2024.07.22 12:10:14.740086 [ 1822 ] {ce9c833c-0e9d-4e85-934e-6c0f68c4b720} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:39032). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002483a9c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002483970f 5. DB::WriteBuffer::next() @ 0x000000001803868c 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x00000000334647a3 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x00000000334447b9 8. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:14.740189 [ 1822 ] {ce9c833c-0e9d-4e85-934e-6c0f68c4b720} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.04416581 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.740324 [ 1822 ] {ce9c833c-0e9d-4e85-934e-6c0f68c4b720} TCPHandler: Client has gone away. 2024.07.22 12:10:14.742552 [ 1606 ] {bf0d98bb-65c0-411d-bfed-289b0f39f05e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.07095375 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57958) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.786715 [ 1606 ] {bf0d98bb-65c0-411d-bfed-289b0f39f05e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.07095375 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:14.787011 [ 1606 ] {bf0d98bb-65c0-411d-bfed-289b0f39f05e} TCPHandler: Client has gone away. 2024.07.22 12:10:15.005158 [ 387 ] {1fb4516a-50d0-46bd-9f8c-ddc2867170b0} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.944975361 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57962) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:15.028228 [ 387 ] {1fb4516a-50d0-46bd-9f8c-ddc2867170b0} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.944975361 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:15.030890 [ 2774 ] {e88a7a54-551f-44c4-a9cd-0168ca78e47f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004021335 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:42896) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:15.033471 [ 387 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:57962). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:15.033888 [ 2773 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:39044). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:15.034636 [ 2774 ] {e88a7a54-551f-44c4-a9cd-0168ca78e47f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004021335 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.363842 [ 2773 ] {2b98071f-9d78-46cb-9656-e9e306d43564} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.091116904 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54060) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.366569 [ 2774 ] {40565832-c712-4a90-a10c-527edc9b034b} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.095146154 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49684) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.411922 [ 2773 ] {2b98071f-9d78-46cb-9656-e9e306d43564} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.091116904 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.413242 [ 2771 ] {bec4a3d4-185d-49eb-ad97-6b4a121f3d42} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.132487914 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39050) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.413464 [ 2775 ] {7f83b22e-2608-4a1f-8318-c144ed3ca68f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.137994837 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45974) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.415561 [ 388 ] {bbf7b8a0-6cf5-4568-8b2f-35b8072c8a28} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.135987512 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42914) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.418246 [ 387 ] {9a8bcc0c-7f3e-45fe-b6d8-bcb9fcaca6a8} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.015703039 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48794) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.419948 [ 2010 ] {23aaf97a-c2bd-4e0e-9e90-fca3957655a6} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.144371281 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57964) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.422847 [ 2774 ] {40565832-c712-4a90-a10c-527edc9b034b} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.095146154 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.431824 [ 2775 ] {7f83b22e-2608-4a1f-8318-c144ed3ca68f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.137994837 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.439411 [ 387 ] {9a8bcc0c-7f3e-45fe-b6d8-bcb9fcaca6a8} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.015703039 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.441802 [ 2010 ] {23aaf97a-c2bd-4e0e-9e90-fca3957655a6} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.144371281 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.449270 [ 388 ] {bbf7b8a0-6cf5-4568-8b2f-35b8072c8a28} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.135987512 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.457494 [ 2771 ] {bec4a3d4-185d-49eb-ad97-6b4a121f3d42} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.132487914 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.458717 [ 2033 ] {e0e56017-aa7b-4779-ad1b-12d5d0fe939b} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.21286549 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:42936) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:16.461222 [ 2771 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:39050). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:16.461941 [ 2033 ] {e0e56017-aa7b-4779-ad1b-12d5d0fe939b} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.21286549 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.843148 [ 2775 ] {4ad8f2e4-b3b8-42c1-a1b4-281c9913a1c7} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.077775271 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39058) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.885573 [ 1683 ] {8d24d1db-3885-42fc-8c4f-72a4e2e08227} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.098246167 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56134) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.893819 [ 2775 ] {4ad8f2e4-b3b8-42c1-a1b4-281c9913a1c7} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.077775271 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.906067 [ 387 ] {d079944c-c814-44d5-8056-3501a045281c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.049892496 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45988) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.912269 [ 2773 ] {3383749a-43d2-40b8-a487-56d5882095ca} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.19530883 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57974) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.919766 [ 2774 ] {128e0a44-0915-4823-9451-bf1598e2d4f3} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.216099471 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48804) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.948552 [ 1683 ] {8d24d1db-3885-42fc-8c4f-72a4e2e08227} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.098246167 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.962336 [ 2774 ] {128e0a44-0915-4823-9451-bf1598e2d4f3} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.216099471 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.985248 [ 2773 ] {3383749a-43d2-40b8-a487-56d5882095ca} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.19530883 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.987768 [ 388 ] {306b25b5-9625-4651-9c4b-9eb37b209634} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.28027207 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54074) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.987907 [ 2033 ] {a98eb315-f80a-453c-a6b3-b784b09773f8} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020021921 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:42948) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.989206 [ 2771 ] {e94ca822-bc5a-42e3-b79e-d125e1df8ffe} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.182985173 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49696) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.991314 [ 2033 ] {a98eb315-f80a-453c-a6b3-b784b09773f8} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020021921 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.991385 [ 2775 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:39058). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:17.991698 [ 2773 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:57974). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:17.995264 [ 2010 ] {ca0ac3a9-d747-4590-a1b0-035c4de737eb} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.182093203 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42926) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.996767 [ 387 ] {d079944c-c814-44d5-8056-3501a045281c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.049892496 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:17.997020 [ 387 ] {d079944c-c814-44d5-8056-3501a045281c} TCPHandler: Client has gone away. 2024.07.22 12:10:18.009163 [ 2771 ] {e94ca822-bc5a-42e3-b79e-d125e1df8ffe} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.182985173 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:18.009542 [ 2771 ] {e94ca822-bc5a-42e3-b79e-d125e1df8ffe} TCPHandler: Client has gone away. 2024.07.22 12:10:18.012210 [ 388 ] {306b25b5-9625-4651-9c4b-9eb37b209634} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.28027207 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:18.012550 [ 388 ] {306b25b5-9625-4651-9c4b-9eb37b209634} TCPHandler: Client has gone away. 2024.07.22 12:10:18.016281 [ 2010 ] {ca0ac3a9-d747-4590-a1b0-035c4de737eb} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.182093203 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:18.016497 [ 2010 ] {ca0ac3a9-d747-4590-a1b0-035c4de737eb} TCPHandler: Client has gone away. 2024.07.22 12:10:19.423469 [ 1822 ] {15ab6e19-797d-43a0-bf66-79c6e4d6e75b} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.222274948 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49704) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.428073 [ 2775 ] {1277415e-537f-4740-bf82-2faba45afe7c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020022025 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42938) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.434125 [ 1606 ] {ec75a0a0-b313-4d90-b3b1-cd69722f982a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.112024011 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56138) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.442338 [ 1822 ] {15ab6e19-797d-43a0-bf66-79c6e4d6e75b} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.222274948 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.449462 [ 2775 ] {1277415e-537f-4740-bf82-2faba45afe7c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020022025 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.453806 [ 1606 ] {ec75a0a0-b313-4d90-b3b1-cd69722f982a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.112024011 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.468016 [ 387 ] {ee433224-f1f7-43a9-a6db-25befb97f6e4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.237637569 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37174) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.473388 [ 2771 ] {ed32d3b6-a817-41c1-b110-831163adedd9} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.092023587 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45992) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.481968 [ 2774 ] {10189f9e-913c-47ba-bc58-c3508a157810} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.254850706 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54084) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.491261 [ 387 ] {ee433224-f1f7-43a9-a6db-25befb97f6e4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.237637569 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.492809 [ 2771 ] {ed32d3b6-a817-41c1-b110-831163adedd9} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.092023587 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.503394 [ 2774 ] {10189f9e-913c-47ba-bc58-c3508a157810} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.254850706 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.503422 [ 2033 ] {268a3ddc-7beb-472e-8afb-ebfc83188189} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.316675126 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39068) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.517911 [ 2773 ] {3339067c-dbf4-409b-91b0-a49f31271780} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.024022117 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57976) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.521477 [ 2033 ] {268a3ddc-7beb-472e-8afb-ebfc83188189} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.316675126 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.525312 [ 1683 ] {ed900947-fc50-4422-9709-29592039fc87} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000021586 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:42952) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.530328 [ 1683 ] {ed900947-fc50-4422-9709-29592039fc87} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000021586 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.530433 [ 2033 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:39068). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:19.544034 [ 2773 ] {3339067c-dbf4-409b-91b0-a49f31271780} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.024022117 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:19.544329 [ 2773 ] {3339067c-dbf4-409b-91b0-a49f31271780} TCPHandler: Client has gone away. 2024.07.22 12:10:21.032496 [ 2775 ] {83247898-e92b-412d-8888-9134fd9d703e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.26675311 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:46004) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.050973 [ 2775 ] {83247898-e92b-412d-8888-9134fd9d703e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.26675311 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.061144 [ 388 ] {ab24fe3b-9063-4a3d-ad39-153073a8e452} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.164025251 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37190) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.067389 [ 2033 ] {b9c17330-4243-45b0-a15e-1468e1a8b33e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.329347198 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39078) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.068086 [ 1822 ] {ef33222e-3406-428f-918d-c7d730a1b0c8} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.290270773 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57978) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.078539 [ 387 ] {e9a2029d-5230-4a63-9d4e-69f634e0d8a1} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.072023245 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:43608) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.079982 [ 388 ] {ab24fe3b-9063-4a3d-ad39-153073a8e452} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.164025251 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.092420 [ 1822 ] {ef33222e-3406-428f-918d-c7d730a1b0c8} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.290270773 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.093483 [ 2033 ] {b9c17330-4243-45b0-a15e-1468e1a8b33e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.329347198 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.098258 [ 2771 ] {42c2a2cf-a856-43b5-a8a2-02d3879b42dd} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.32963049 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56154) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.100935 [ 387 ] {e9a2029d-5230-4a63-9d4e-69f634e0d8a1} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.072023245 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.102742 [ 1606 ] {dc761a26-9baf-460d-bcd8-ebf0a30c7a2d} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.136024647 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49720) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.105041 [ 3602 ] {aacac283-3565-4489-9579-d9d73efa8d74} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.338276469 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54096) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.110527 [ 2774 ] {f9f9358d-6bd3-4bc8-9e31-2866e987597c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.357631561 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42954) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.128165 [ 2771 ] {42c2a2cf-a856-43b5-a8a2-02d3879b42dd} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.32963049 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.130151 [ 3602 ] {aacac283-3565-4489-9579-d9d73efa8d74} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.338276469 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.136191 [ 1606 ] {dc761a26-9baf-460d-bcd8-ebf0a30c7a2d} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.136024647 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3dca1a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000033529699 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x0000000033517cbb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000033518865 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000002473d9db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024742a34 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 13. ? @ 0x00007fb3e4339ac3 14. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.138695 [ 2774 ] {f9f9358d-6bd3-4bc8-9e31-2866e987597c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.357631561 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.267156 [ 2773 ] {1ebbe452-1583-416e-b455-94d66ed24b98} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.406892674 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48818) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:103: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000033515660 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 10. ? @ 0x00007fb3e4339ac3 11. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.267202 [ 1683 ] {4ddd630f-5b5d-4a71-acb8-bd090e126139} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.555490979 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:42954) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.270122 [ 1606 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:49720). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:21.270296 [ 3602 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54096). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:21.270617 [ 2033 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:39078). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000024835ee7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024834cef 5. DB::ReadBuffer::next() @ 0x0000000018735150 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033442501 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x000000003347e3d6 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003676223e 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676335a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369cf170 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369ca271 12. ? @ 0x00007fb3e4339ac3 13. ? @ 0x00007fb3e43cb850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:10:21.270719 [ 1683 ] {4ddd630f-5b5d-4a71-acb8-bd090e126139} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.555490979 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.279668 [ 2773 ] {1ebbe452-1583-416e-b455-94d66ed24b98} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.406892674 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:103: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000033515660 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 10. ? @ 0x00007fb3e4339ac3 11. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.279941 [ 2773 ] {1ebbe452-1583-416e-b455-94d66ed24b98} TCPHandler: Client has gone away. 2024.07.22 12:10:21.355022 [ 2010 ] {07c7b18c-c2af-457c-b952-63be8a9f7633} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.107415277 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48808) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.378510 [ 2010 ] {07c7b18c-c2af-457c-b952-63be8a9f7633} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.107415277 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x000000001819c375 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x000000003691cd85 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000002462f4cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3de082 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3dd9fe 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x0000000033515954 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x0000000033515bb9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003351575a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003352ed31 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002473b03b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000002473fd8a 11. ? @ 0x00007fb3e4339ac3 12. ? @ 0x00007fb3e43cb850 2024.07.22 12:10:21.378714 [ 2010 ] {07c7b18c-c2af-457c-b952-63be8a9f7633} TCPHandler: Client has gone away. 2024.07.22 12:12:29.141439 [ 384 ] {} Application: Closed all listening sockets. Waiting for 4 outstanding connections. 2024.07.22 12:12:34.046792 [ 384 ] {} Application: Closed connections. But 4 remain. Tip: To increase wait time add to config: 60 2024.07.22 12:12:34.046862 [ 384 ] {} Application: Will shutdown forcefully.