2024.11.04 17:31:38.794956 [ 391 ] {} Context: Server was built with sanitizer. It will work slowly. 2024.11.04 17:31:38.805429 [ 391 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2024.11.04 17:31:38.805976 [ 391 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.11.04 17:31:39.744994 [ 391 ] {} CloudPlacementInfo: Placement info has not been loaded 2024.11.04 17:31:39.878586 [ 391 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2024.11.04 17:31:39.879807 [ 391 ] {} KeeperLogStore: Removing all changelogs 2024.11.04 17:31:39.880502 [ 391 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.11.04 17:31:39.887543 [ 391 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.11.04 17:31:39.887566 [ 391 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.11.04 17:31:39.938163 [ 972 ] {} RaftInstance: Election timeout, initiate leader election 2024.11.04 17:31:39.959480 [ 391 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2024.11.04 17:31:39.959515 [ 391 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2024.11.04 17:31:40.060147 [ 391 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.11.04 17:31:41.041755 [ 393 ] {d675b59c-9038-4b64-a7ce-d5d5e88164f3} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.11.04 17:31:49.412321 [ 393 ] {ad6ce99d-e98f-4103-b766-f41fe3fe6a20} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.11.04 17:33:51.509083 [ 395 ] {cc1458ec-ed9f-4e7d-a97d-650036b75c63} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37606) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335cc919 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033605f12 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003354cf52 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ac4f3 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5c667a 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5c5e47 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbfdd3e 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba5912 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba2d87 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbdf143 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba50e5 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba2d87 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6901: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbbd85a 17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db8c6c4 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000002db85b57 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002db7cc0e 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002e5c41f6 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000002e5bcd14 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000002e5c7e41 23. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002e478c3d 24. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1fe6aa 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1f5a4a 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326e1fbd 27. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 2024.11.04 17:33:51.513334 [ 395 ] {cc1458ec-ed9f-4e7d-a97d-650036b75c63} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335cc919 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033605f12 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003354cf52 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ac4f3 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5c667a 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5c5e47 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbfdd3e 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba5912 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba2d87 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbdf143 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba50e5 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba2d87 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6901: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbbd85a 17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db8c6c4 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000002db85b57 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002db7cc0e 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002e5c41f6 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000002e5bcd14 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000002e5c7e41 23. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002e478c3d 24. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1fe6aa 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1f5a4a 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326e1fbd 27. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 2024.11.04 17:33:51.652739 [ 395 ] {f0f6354b-342f-4d79-9b18-96a60490db68} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37606) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335cc919 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033605f12 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003354cf52 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ac4f3 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5c667a 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5c5e47 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1ffbe7 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1f5a4a 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326e1fbd 13. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 18. asan_thread_start(void*) @ 0x000000000a7c5c1b 19. ? @ 0x00007efeee467ac3 20. ? @ 0x00007efeee4f9850 2024.11.04 17:33:51.655172 [ 395 ] {f0f6354b-342f-4d79-9b18-96a60490db68} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335cc919 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033605f12 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003354cf52 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ac4f3 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5c667a 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5c5e47 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1ffbe7 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1f5a4a 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326e1fbd 13. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 18. asan_thread_start(void*) @ 0x000000000a7c5c1b 19. ? @ 0x00007efeee467ac3 20. ? @ 0x00007efeee4f9850 2024.11.04 17:33:52.207212 [ 395 ] {950656a4-c38c-4704-8a25-011e0a1409ea} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 16.93 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37606) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:33:52.222431 [ 395 ] {950656a4-c38c-4704-8a25-011e0a1409ea} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 16.93 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:33:52.305839 [ 395 ] {14d6ea77-9bf5-46b4-bda8-751245843fef} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37606) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335cc919 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033605f12 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003354cf52 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ac4f3 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5c667a 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5c5e47 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbfdd3e 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba5912 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba2d87 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbdf143 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba50e5 15. ./contrib/llvm-project/libcxx/include/vector:580: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db8cbd4 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000002db85b57 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002db7cc0e 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002e5c41f6 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000002e5bcd14 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000002e5c7e41 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002e478c3d 22. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1fe6aa 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1f5a4a 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326e1fbd 25. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 30. asan_thread_start(void*) @ 0x000000000a7c5c1b 31. ? @ 0x00007efeee467ac3 2024.11.04 17:33:52.306463 [ 395 ] {14d6ea77-9bf5-46b4-bda8-751245843fef} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335cc919 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033605f12 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003354cf52 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ac4f3 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5c667a 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5c5e47 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbfdd3e 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba5912 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba2d87 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbdf143 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002dba50e5 15. ./contrib/llvm-project/libcxx/include/vector:580: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db8cbd4 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000002db85b57 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002db7cc0e 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002e5c41f6 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000002e5bcd14 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000002e5c7e41 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002e478c3d 22. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1fe6aa 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1f5a4a 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326e1fbd 25. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 30. asan_thread_start(void*) @ 0x000000000a7c5c1b 31. ? @ 0x00007efeee467ac3 2024.11.04 17:33:52.752389 [ 395 ] {d617e35a-7dfe-4995-a058-0e85e94e9230} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.18 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37606) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:33:52.753501 [ 395 ] {d617e35a-7dfe-4995-a058-0e85e94e9230} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.18 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b425279 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424a25 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:37.993036 [ 1626 ] {bbc5fe3f-8e2f-44f2-91ae-2d7b9920fb1f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.150942356 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40074) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:37.992975 [ 2044 ] {b3a1359a-6276-45d1-b9b5-1e3697140b7a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.27461612 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57022) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:38.064328 [ 2044 ] {b3a1359a-6276-45d1-b9b5-1e3697140b7a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.27461612 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:38.114546 [ 1626 ] {bbc5fe3f-8e2f-44f2-91ae-2d7b9920fb1f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.150942356 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:38.136272 [ 1626 ] {bbc5fe3f-8e2f-44f2-91ae-2d7b9920fb1f} TCPHandler: Client has gone away. 2024.11.04 17:37:38.244937 [ 3025 ] {468524b2-4b6a-4912-97e2-df01175d625d} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.16 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57004) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4258be 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424bda 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:38.281337 [ 3025 ] {468524b2-4b6a-4912-97e2-df01175d625d} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.16 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4258be 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424bda 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:39.445963 [ 2043 ] {9049ae84-9aa1-4603-96ee-91b04b501e24} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.017404907 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40080) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:39.477147 [ 2043 ] {9049ae84-9aa1-4603-96ee-91b04b501e24} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.017404907 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:39.592245 [ 1626 ] {be140bf2-3ef2-492a-afa8-13b3dd79c4fe} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.169572905 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56686) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:39.597274 [ 1626 ] {be140bf2-3ef2-492a-afa8-13b3dd79c4fe} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.169572905 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:39.621377 [ 2044 ] {92d64a85-2193-4074-8cb8-e6293a3e18a0} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.216340905 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57036) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:39.679218 [ 2044 ] {92d64a85-2193-4074-8cb8-e6293a3e18a0} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.216340905 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:39.680656 [ 1626 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:56686). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:39.683846 [ 2043 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:40080). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:40.101729 [ 3025 ] {602f8283-5be8-4acd-b08d-6a3a655779d6} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 576.15 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57004) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4258be 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424bda 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:40.302637 [ 3025 ] {602f8283-5be8-4acd-b08d-6a3a655779d6} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 576.15 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4258be 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424bda 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.112736 [ 2043 ] {4e8c7543-051a-48a4-b4aa-48372a7f6e36} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004150765 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59244) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.123934 [ 2043 ] {4e8c7543-051a-48a4-b4aa-48372a7f6e36} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004150765 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.142543 [ 2044 ] {9b46090d-c1f4-4a1a-8d51-d6f97ca85a13} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.132044182 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47134) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.142891 [ 1626 ] {241bf6b5-dc39-4069-9839-c181a277c553} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028529042 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35006) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.148328 [ 1626 ] {241bf6b5-dc39-4069-9839-c181a277c553} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028529042 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.154153 [ 3018 ] {12927421-17a0-4113-a5ed-f4f702355bbe} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.034857406 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38134) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.172975 [ 1626 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:35006). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:41.180221 [ 2044 ] {9b46090d-c1f4-4a1a-8d51-d6f97ca85a13} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.132044182 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.337367 [ 3018 ] {12927421-17a0-4113-a5ed-f4f702355bbe} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:38134). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001ceff0bb 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001cefd042 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003272666c 5. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000032703728 6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000326e4207 7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:41.339361 [ 3018 ] {12927421-17a0-4113-a5ed-f4f702355bbe} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.034857406 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.343943 [ 3018 ] {12927421-17a0-4113-a5ed-f4f702355bbe} TCPHandler: Client has gone away. 2024.11.04 17:37:41.492966 [ 3025 ] {3edc5319-64db-4693-9f1a-e9022b2316c6} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 587.55 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57004) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4258be 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424bda 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:41.537910 [ 3025 ] {3edc5319-64db-4693-9f1a-e9022b2316c6} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 587.55 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4258be 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b424bda 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3dffbf 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.534834 [ 3019 ] {9674c9d2-796b-4160-b8a9-75b157daf0a9} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000885139 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54866) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.539886 [ 3019 ] {9674c9d2-796b-4160-b8a9-75b157daf0a9} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000885139 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.629788 [ 2044 ] {8f7520d1-4f57-433f-8579-5b894d4ba56c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.150732808 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47136) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.629992 [ 2043 ] {1618783d-98e2-4f2b-aebc-573d43ad75e1} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.095612242 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38138) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.635690 [ 3018 ] {10ca5aae-a732-4dd7-836b-68ae75afc88d} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.096090744 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59258) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.643510 [ 3018 ] {10ca5aae-a732-4dd7-836b-68ae75afc88d} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.096090744 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.647315 [ 1626 ] {45965845-64ef-4bd4-811c-c8657c7890a2} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.112008392 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35010) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.649405 [ 2043 ] {1618783d-98e2-4f2b-aebc-573d43ad75e1} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.095612242 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.652014 [ 3018 ] {10ca5aae-a732-4dd7-836b-68ae75afc88d} TCPHandler: Client has gone away. 2024.11.04 17:37:42.670495 [ 1626 ] {45965845-64ef-4bd4-811c-c8657c7890a2} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.112008392 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.673361 [ 2043 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:38138). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:42.674291 [ 2044 ] {8f7520d1-4f57-433f-8579-5b894d4ba56c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.150732808 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:42.674334 [ 1626 ] {45965845-64ef-4bd4-811c-c8657c7890a2} TCPHandler: Client has gone away. 2024.11.04 17:37:44.053349 [ 3019 ] {fd1483ca-1df7-452c-80da-90596437340e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.002114022 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59270) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.059337 [ 2043 ] {25974b14-c6dd-438a-8c33-0e102372bd13} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.022104926 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54872) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.059781 [ 3019 ] {fd1483ca-1df7-452c-80da-90596437340e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.002114022 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.060940 [ 3020 ] {8f157a76-0506-4236-a049-d454dd3a3272} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00761713 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45584) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.066086 [ 2043 ] {25974b14-c6dd-438a-8c33-0e102372bd13} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.022104926 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.072750 [ 1626 ] {b1cb8eb2-29a9-40a7-9935-70be90766776} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.031041262 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35026) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.078273 [ 3018 ] {dfc23a16-7599-4da1-87be-524e2e95ab3e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.0255846 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38152) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.079889 [ 3020 ] {8f157a76-0506-4236-a049-d454dd3a3272} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00761713 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.081832 [ 2044 ] {56735fe2-bfcc-4e2c-8639-9949bd628745} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.114234846 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47160) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.083206 [ 3018 ] {dfc23a16-7599-4da1-87be-524e2e95ab3e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.0255846 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.092708 [ 3020 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:45584). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:44.093271 [ 2043 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54872). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:44.094537 [ 3019 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:59270). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:44.099157 [ 3018 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:38152). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:44.100087 [ 2044 ] {56735fe2-bfcc-4e2c-8639-9949bd628745} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.114234846 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.101976 [ 1626 ] {b1cb8eb2-29a9-40a7-9935-70be90766776} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.031041262 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:44.109012 [ 1626 ] {b1cb8eb2-29a9-40a7-9935-70be90766776} TCPHandler: Client has gone away. 2024.11.04 17:37:45.486028 [ 3018 ] {e2e24928-b987-457d-97f7-4beebeb7780e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.013186119 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45588) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.496619 [ 3022 ] {6d85ad4e-45f1-4e1c-939b-30b800d1a31a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.029650307 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54876) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.505650 [ 3023 ] {78bbe0a4-ceb9-4634-90a4-5a2798944456} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.030345598 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35042) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.508309 [ 3018 ] {e2e24928-b987-457d-97f7-4beebeb7780e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.013186119 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.508659 [ 3020 ] {66f4abf8-f578-4f18-baca-52b734bfe2aa} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.041264388 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:46972) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.509553 [ 3023 ] {78bbe0a4-ceb9-4634-90a4-5a2798944456} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.030345598 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.513600 [ 3020 ] {66f4abf8-f578-4f18-baca-52b734bfe2aa} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.041264388 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.527369 [ 3022 ] {6d85ad4e-45f1-4e1c-939b-30b800d1a31a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.029650307 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.529717 [ 2043 ] {88b0212f-1734-4515-a7c6-c247e618d5d0} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.128324217 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47162) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.531639 [ 3021 ] {f86f1c35-1d55-4d2f-823f-4fc2fdc229ef} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064610912 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59272) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.535611 [ 3021 ] {f86f1c35-1d55-4d2f-823f-4fc2fdc229ef} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064610912 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.545246 [ 3019 ] {8b86a2f6-f723-4f62-882a-a4623834d3e2} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.084541301 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38162) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.584133 [ 3022 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54876). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:45.587990 [ 3021 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:59272). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:45.592115 [ 3023 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:35042). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:45.596809 [ 2043 ] {88b0212f-1734-4515-a7c6-c247e618d5d0} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.128324217 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.837655 [ 3019 ] {8b86a2f6-f723-4f62-882a-a4623834d3e2} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.084541301 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:45.838457 [ 3019 ] {8b86a2f6-f723-4f62-882a-a4623834d3e2} TCPHandler: Client has gone away. 2024.11.04 17:37:47.036918 [ 3018 ] {66f7a22b-b5dd-4405-8a6a-d8ab6f6ca8df} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004932827 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45592) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.046176 [ 3020 ] {909ecc50-613e-425b-84b5-213f33c7aa77} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.015723947 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38172) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.070834 [ 3018 ] {66f7a22b-b5dd-4405-8a6a-d8ab6f6ca8df} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004932827 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.070477 [ 2043 ] {545cd3cf-1328-46c2-8726-db999ba19ab4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004008944 seconds, maximum: 1: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47178) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3df857 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.072521 [ 3023 ] {836cf1d7-2e1b-4b6c-80da-695a39916543} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.039333659 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54880) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.080268 [ 3023 ] {836cf1d7-2e1b-4b6c-80da-695a39916543} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.039333659 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.084656 [ 3020 ] {909ecc50-613e-425b-84b5-213f33c7aa77} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.015723947 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.084890 [ 3024 ] {83cd7613-89e8-4624-838d-4745838280e1} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016009052 seconds, maximum: 1: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35052) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3df857 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.087203 [ 3021 ] {ff73e4ac-fe08-492b-aab2-2c99d4fd2382} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.059815396 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59282) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.091386 [ 3021 ] {ff73e4ac-fe08-492b-aab2-2c99d4fd2382} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.059815396 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.094776 [ 3019 ] {9d570494-6a0c-4926-9af0-df9cf45a276f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.072924112 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:46986) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.105232 [ 3024 ] {83cd7613-89e8-4624-838d-4745838280e1} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016009052 seconds, maximum: 1: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3df857 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.112075 [ 3018 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:45592). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:47.113365 [ 3019 ] {9d570494-6a0c-4926-9af0-df9cf45a276f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.072924112 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.114217 [ 3023 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54880). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:47.115965 [ 3019 ] {9d570494-6a0c-4926-9af0-df9cf45a276f} TCPHandler: Client has gone away. 2024.11.04 17:37:47.116031 [ 3021 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:59282). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:47.123636 [ 3022 ] {20f18119-ab40-4c63-98ca-f8f97db51706} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.091091249 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44224) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.123833 [ 2043 ] {545cd3cf-1328-46c2-8726-db999ba19ab4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004008944 seconds, maximum: 1: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3df857 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003285eab9 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003283f543 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000032840766 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd6119d 9. ./src/Common/ThreadPool.h:226: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd6b253 10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 12. asan_thread_start(void*) @ 0x000000000a7c5c1b 13. ? @ 0x00007efeee467ac3 14. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.126995 [ 3020 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:38172). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:47.128753 [ 3024 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:35052). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:47.432148 [ 3022 ] {20f18119-ab40-4c63-98ca-f8f97db51706} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.091091249 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:47.432472 [ 3022 ] {20f18119-ab40-4c63-98ca-f8f97db51706} TCPHandler: Client has gone away. 2024.11.04 17:37:48.535279 [ 3022 ] {147ceae8-c3b5-4cce-961f-07e301cadc6e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00159696 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48258) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.535327 [ 3023 ] {d9350b5c-6642-4f21-b4ce-8ccca08c7260} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.003457546 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54882) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.540535 [ 3022 ] {147ceae8-c3b5-4cce-961f-07e301cadc6e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00159696 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.542415 [ 3019 ] {1d0083fe-6835-4ee9-a1b6-ae1e7e3bd84a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.010386449 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:46996) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.546804 [ 3023 ] {d9350b5c-6642-4f21-b4ce-8ccca08c7260} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.003457546 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.547133 [ 3020 ] {5c205657-1ecf-47f0-8187-d62cdc15c745} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.010500465 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44236) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.552134 [ 3020 ] {5c205657-1ecf-47f0-8187-d62cdc15c745} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.010500465 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.555019 [ 3021 ] {1850c85d-638a-4f67-9c80-55ef574ea52a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.025051599 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45602) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.556173 [ 3018 ] {a0dbf629-96f2-4e54-80de-9dd87267ed28} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016207375 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38176) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.560888 [ 3018 ] {a0dbf629-96f2-4e54-80de-9dd87267ed28} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016207375 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.561372 [ 3021 ] {1850c85d-638a-4f67-9c80-55ef574ea52a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.025051599 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.562969 [ 3026 ] {939008c1-d920-4ee6-ac84-b7ca0ed0fefd} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.027383532 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59298) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.587635 [ 3026 ] {939008c1-d920-4ee6-ac84-b7ca0ed0fefd} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.027383532 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.588345 [ 3019 ] {1d0083fe-6835-4ee9-a1b6-ae1e7e3bd84a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.010386449 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.604596 [ 3024 ] {1d5e2f0f-e3ea-42cd-accb-a4db31349921} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.069292068 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35054) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.606333 [ 2043 ] {e2ba92c1-d071-4fd0-ba5a-8f4f917a4bde} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.15417597 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47182) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.608709 [ 3024 ] {1d5e2f0f-e3ea-42cd-accb-a4db31349921} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.069292068 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.623651 [ 3019 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:46996). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:48.634246 [ 3021 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:45602). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:48.642518 [ 3023 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54882). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:48.648478 [ 3026 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:59298). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:48.653983 [ 3024 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:35054). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:48.655937 [ 2043 ] {e2ba92c1-d071-4fd0-ba5a-8f4f917a4bde} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.15417597 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:48.657892 [ 3018 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:38176). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cef8d16 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cef7072 4. DB::ReadBuffer::next() @ 0x000000000b52d5eb 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326e09eb 6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x0000000032726a11 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a2120af 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a212d57 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a61fd3c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a618d28 11. asan_thread_start(void*) @ 0x000000000a7c5c1b 12. ? @ 0x00007efeee467ac3 13. ? @ 0x00007efeee4f9850 (version 24.3.12.76.altinitystable (altinity build)) 2024.11.04 17:37:50.028938 [ 2043 ] {702fe070-4007-4deb-99aa-35e3fbd61424} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.083369797 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47186) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:37:50.252159 [ 2043 ] {702fe070-4007-4deb-99aa-35e3fbd61424} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.083369797 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a509d73 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb535b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3e313d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3e213f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e85fc11 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003283bfd5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003283b661 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000328687e5 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001cd581a0 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd653ad 10. asan_thread_start(void*) @ 0x000000000a7c5c1b 11. ? @ 0x00007efeee467ac3 12. ? @ 0x00007efeee4f9850 2024.11.04 17:41:55.999828 [ 391 ] {} Application: Closed all listening sockets. Waiting for 4 outstanding connections.