2024.07.22 11:59:08.877009 [ 390 ] {} Context: Server was built with sanitizer. It will work slowly. 2024.07.22 11:59:08.888836 [ 390 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2024.07.22 11:59:08.889383 [ 390 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.07.22 11:59:09.697479 [ 390 ] {} CloudPlacementInfo: Placement info has not been loaded 2024.07.22 11:59:09.809725 [ 390 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2024.07.22 11:59:09.810923 [ 390 ] {} KeeperLogStore: Removing all changelogs 2024.07.22 11:59:09.811864 [ 390 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.07.22 11:59:09.817544 [ 390 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.07.22 11:59:09.817576 [ 390 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.07.22 11:59:09.868186 [ 973 ] {} RaftInstance: Election timeout, initiate leader election 2024.07.22 11:59:09.889043 [ 390 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2024.07.22 11:59:09.889072 [ 390 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2024.07.22 11:59:09.988788 [ 390 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.07.22 11:59:10.960740 [ 392 ] {ce0010b6-f9b8-4ce4-b21e-70efa221c823} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.07.22 11:59:19.349799 [ 392 ] {f164e409-b911-468f-b204-71456c075816} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.07.22 12:01:24.011008 [ 392 ] {06918639-9987-444c-9777-97e18630c3aa} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60734) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335b0e79 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ea472 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003353b8b2 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033590a93 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5a0eda 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5a06a7 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dc1321e 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db81032 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db7e4a7 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5546: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbb6643 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db80810 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db7e4a7 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6880: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db94d9a 17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db67de4 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000002db61277 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002db5832e 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002e59ea56 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000002e597574 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000002e5a26a1 23. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002e45063d 24. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1cf58a 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1c692a 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326d027d 27. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 2024.07.22 12:01:24.011491 [ 392 ] {06918639-9987-444c-9777-97e18630c3aa} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335b0e79 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ea472 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003353b8b2 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033590a93 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5a0eda 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5a06a7 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dc1321e 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db81032 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db7e4a7 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5546: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbb6643 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db80810 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db7e4a7 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6880: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db94d9a 17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db67de4 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000002db61277 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002db5832e 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002e59ea56 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000002e597574 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000002e5a26a1 23. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002e45063d 24. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1cf58a 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1c692a 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326d027d 27. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 2024.07.22 12:01:24.066323 [ 392 ] {3a43eb5b-b3cc-48ec-b0b5-df11733ac982} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60734) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335b0e79 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ea472 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003353b8b2 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033590a93 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5a0eda 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5a06a7 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1d0ac7 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1c692a 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326d027d 13. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 18. asan_thread_start(void*) @ 0x000000000a7b5a5b 19. ? @ 0x00007f2ff11fbac3 20. ? @ 0x00007f2ff128d850 2024.07.22 12:01:24.066783 [ 392 ] {3a43eb5b-b3cc-48ec-b0b5-df11733ac982} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335b0e79 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ea472 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003353b8b2 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033590a93 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5a0eda 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5a06a7 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1d0ac7 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1c692a 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326d027d 13. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 18. asan_thread_start(void*) @ 0x000000000a7b5a5b 19. ? @ 0x00007f2ff11fbac3 20. ? @ 0x00007f2ff128d850 2024.07.22 12:01:24.370870 [ 392 ] {ea053824-c831-4449-822e-2471662a7751} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 16.83 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60734) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:01:24.401023 [ 392 ] {ea053824-c831-4449-822e-2471662a7751} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 16.83 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:01:24.489222 [ 392 ] {8b4b00f0-e99a-4704-b419-c08255d00e4b} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60734) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335b0e79 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ea472 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003353b8b2 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033590a93 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5a0eda 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5a06a7 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dc1321e 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db81032 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db7e4a7 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5546: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbb6643 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db80810 15. ./contrib/llvm-project/libcxx/include/vector:580: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db682f4 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000002db61277 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002db5832e 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002e59ea56 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000002e597574 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000002e5a26a1 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002e45063d 22. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1cf58a 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1c692a 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326d027d 25. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 30. asan_thread_start(void*) @ 0x000000000a7b5a5b 31. ? @ 0x00007f2ff11fbac3 2024.07.22 12:01:24.489698 [ 392 ] {8b4b00f0-e99a-4704-b419-c08255d00e4b} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x00000000335b0e79 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000335ea472 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003353b8b2 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033590a93 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002e5a0eda 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000002e5a06a7 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dc1321e 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db81032 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db7e4a7 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5546: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002dbb6643 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000002db80810 15. ./contrib/llvm-project/libcxx/include/vector:580: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000002db682f4 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000002db61277 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002db5832e 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002e59ea56 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000002e597574 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000002e5a26a1 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002e45063d 22. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002f1cf58a 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002f1c692a 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000326d027d 25. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 30. asan_thread_start(void*) @ 0x000000000a7b5a5b 31. ? @ 0x00007f2ff11fbac3 2024.07.22 12:01:24.562677 [ 392 ] {196ed286-df1f-4462-9427-7227cb3fb79e} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60734) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:01:24.565673 [ 392 ] {196ed286-df1f-4462-9427-7227cb3fb79e} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002b4051b9 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404965 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:04:54.496195 [ 2954 ] {a1c0ba70-5fc0-44ff-ad6b-ac0e2f1016ad} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 591.32 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:56558) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4057fe 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404b1a 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:04:54.540450 [ 2954 ] {a1c0ba70-5fc0-44ff-ad6b-ac0e2f1016ad} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 591.32 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4057fe 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404b1a 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:04:56.011111 [ 1989 ] {073bbef4-d453-4048-9473-301c6776092d} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.231726183 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57368) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:04:56.123968 [ 2956 ] {5f5ce3fa-31dd-44ca-991d-bddb05fa4050} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.874591211 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:56582) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:04:56.745979 [ 1989 ] {073bbef4-d453-4048-9473-301c6776092d} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.231726183 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:04:56.757904 [ 1989 ] {073bbef4-d453-4048-9473-301c6776092d} TCPHandler: Client has gone away. 2024.07.22 12:04:56.907801 [ 2954 ] {edbd8673-3c99-4509-9610-0519395be834} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.46 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:56558) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4057fe 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404b1a 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:04:56.943275 [ 2956 ] {5f5ce3fa-31dd-44ca-991d-bddb05fa4050} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.874591211 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:04:56.961197 [ 2954 ] {edbd8673-3c99-4509-9610-0519395be834} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.46 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4057fe 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404b1a 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:04:59.416562 [ 1984 ] {23a3d2c5-1c96-4fb9-b519-39bb729d151f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.367070181 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51264) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:04:59.696661 [ 2954 ] {7f9efba6-1320-4994-b533-e243e2498387} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 579.55 MiB: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:56558) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4057fe 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404b1a 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:04:59.723647 [ 1989 ] {56bc8cb7-7fe3-4be5-a921-50398526cb8c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.931360177 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45302) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:04:59.728985 [ 2956 ] {60dfdd67-6330-41af-afc9-9657aeca85c5} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.457587044 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60964) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:04:59.757924 [ 2954 ] {7f9efba6-1320-4994-b533-e243e2498387} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 579.55 MiB: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002b4057fe 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002b404b1a 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bfeff 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:04:59.759363 [ 1984 ] {23a3d2c5-1c96-4fb9-b519-39bb729d151f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.367070181 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:04:59.759855 [ 1984 ] {23a3d2c5-1c96-4fb9-b519-39bb729d151f} TCPHandler: Client has gone away. 2024.07.22 12:04:59.800577 [ 1989 ] {56bc8cb7-7fe3-4be5-a921-50398526cb8c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.931360177 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:04:59.800915 [ 1989 ] {56bc8cb7-7fe3-4be5-a921-50398526cb8c} TCPHandler: Client has gone away. 2024.07.22 12:04:59.827944 [ 2956 ] {60dfdd67-6330-41af-afc9-9657aeca85c5} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.457587044 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:01.878001 [ 1984 ] {fabf879a-fa9e-4cfe-8749-30f469d78b03} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.184184537 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45860) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:01.886370 [ 1987 ] {37c241dd-9280-4385-a2c0-afae5fdede21} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.06359094 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51268) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:01.895632 [ 2956 ] {5d3de0a5-dadb-48b1-88b5-2c0deb2c3253} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.768740377 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60992) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:01.936154 [ 1989 ] {00a26f4b-c8ab-4a02-9c26-373fb6f04042} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.293621069 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45312) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:02.434091 [ 1987 ] {37c241dd-9280-4385-a2c0-afae5fdede21} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.06359094 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:02.437814 [ 1987 ] {37c241dd-9280-4385-a2c0-afae5fdede21} TCPHandler: Client has gone away. 2024.07.22 12:05:02.542626 [ 1984 ] {fabf879a-fa9e-4cfe-8749-30f469d78b03} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.184184537 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:02.549192 [ 1984 ] {fabf879a-fa9e-4cfe-8749-30f469d78b03} TCPHandler: Client has gone away. 2024.07.22 12:05:02.674262 [ 1989 ] {00a26f4b-c8ab-4a02-9c26-373fb6f04042} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:45312). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001cee55fb 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001cee3582 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x00000000327142cc 5. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x00000000326f19e8 6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000326d24c7 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:05:02.674498 [ 1989 ] {00a26f4b-c8ab-4a02-9c26-373fb6f04042} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.293621069 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:02.677674 [ 1989 ] {00a26f4b-c8ab-4a02-9c26-373fb6f04042} TCPHandler: Client has gone away. 2024.07.22 12:05:03.040048 [ 2956 ] {5d3de0a5-dadb-48b1-88b5-2c0deb2c3253} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.768740377 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:05.189881 [ 1984 ] {6ea08b09-133f-48f9-91d6-45788f44ef8f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.271959876 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45874) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:05.594438 [ 1989 ] {4cef21e9-60dd-45d1-89b8-3d87d315629e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.734243885 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45328) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:05.594607 [ 2956 ] {22950ccf-bd66-4177-a2e5-ce37d5f46c22} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.208967513 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:32768) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:05.618219 [ 1987 ] {2ce521c7-2d69-489c-8ae5-9902b2b870ca} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.50762709 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38778) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:05.904120 [ 1984 ] {6ea08b09-133f-48f9-91d6-45788f44ef8f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.271959876 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:05.909132 [ 1984 ] {6ea08b09-133f-48f9-91d6-45788f44ef8f} TCPHandler: Client has gone away. 2024.07.22 12:05:05.948609 [ 1989 ] {4cef21e9-60dd-45d1-89b8-3d87d315629e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.734243885 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:05.951418 [ 1989 ] {4cef21e9-60dd-45d1-89b8-3d87d315629e} TCPHandler: Client has gone away. 2024.07.22 12:05:05.980029 [ 1987 ] {2ce521c7-2d69-489c-8ae5-9902b2b870ca} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.50762709 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:05.980914 [ 1987 ] {2ce521c7-2d69-489c-8ae5-9902b2b870ca} TCPHandler: Client has gone away. 2024.07.22 12:05:06.171437 [ 2956 ] {22950ccf-bd66-4177-a2e5-ce37d5f46c22} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.208967513 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:08.576666 [ 1989 ] {5596903c-32fc-4ae1-8cdb-3511e4929b44} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.427814196 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48786) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003282918f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 9. asan_thread_start(void*) @ 0x000000000a7b5a5b 10. ? @ 0x00007f2ff11fbac3 11. ? @ 0x00007f2ff128d850 2024.07.22 12:05:08.587807 [ 1989 ] {5596903c-32fc-4ae1-8cdb-3511e4929b44} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.427814196 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003282918f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 9. asan_thread_start(void*) @ 0x000000000a7b5a5b 10. ? @ 0x00007f2ff11fbac3 11. ? @ 0x00007f2ff128d850 2024.07.22 12:05:10.262072 [ 1984 ] {a0ae7622-9ce0-4fcf-9acc-1058446ccc0f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.123991324 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50352) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bf797 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:10.262080 [ 1987 ] {55e9158b-8b3c-4b7f-84f7-c2e7d475a26e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.972943965 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:43350) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:10.262086 [ 2956 ] {1226599c-06cc-48a1-bfc3-faec38364bdf} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.710660566 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:50840) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:10.280625 [ 1985 ] {16fc9088-4206-4ca4-8e12-0b3787e517fe} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.867284296 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48746) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:10.329001 [ 1989 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:48786). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cedf256 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cedd5b2 4. DB::ReadBuffer::next() @ 0x000000000b51c50b 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326cecab 6. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 11. asan_thread_start(void*) @ 0x000000000a7b5a5b 12. ? @ 0x00007f2ff11fbac3 13. ? @ 0x00007f2ff128d850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:05:10.429413 [ 1987 ] {55e9158b-8b3c-4b7f-84f7-c2e7d475a26e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.972943965 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:10.431931 [ 1987 ] {55e9158b-8b3c-4b7f-84f7-c2e7d475a26e} TCPHandler: Client has gone away. 2024.07.22 12:05:10.458127 [ 1985 ] {16fc9088-4206-4ca4-8e12-0b3787e517fe} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.867284296 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:10.463841 [ 1985 ] {16fc9088-4206-4ca4-8e12-0b3787e517fe} TCPHandler: Client has gone away. 2024.07.22 12:05:10.544260 [ 1984 ] {a0ae7622-9ce0-4fcf-9acc-1058446ccc0f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.123991324 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bf797 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:10.545819 [ 1984 ] {a0ae7622-9ce0-4fcf-9acc-1058446ccc0f} TCPHandler: Client has gone away. 2024.07.22 12:05:10.691649 [ 2956 ] {1226599c-06cc-48a1-bfc3-faec38364bdf} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.710660566 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:12.937957 [ 1987 ] {0257fe47-612e-463a-883d-6d812d24d159} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016900595 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:43366) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003282918f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 9. asan_thread_start(void*) @ 0x000000000a7b5a5b 10. ? @ 0x00007f2ff11fbac3 11. ? @ 0x00007f2ff128d850 2024.07.22 12:05:12.945972 [ 2956 ] {ea81a92c-deb6-4908-84bc-ce36e217817c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.889622592 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:50852) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:12.948767 [ 1987 ] {0257fe47-612e-463a-883d-6d812d24d159} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016900595 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003282918f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 9. asan_thread_start(void*) @ 0x000000000a7b5a5b 10. ? @ 0x00007f2ff11fbac3 11. ? @ 0x00007f2ff128d850 2024.07.22 12:05:12.995649 [ 1987 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:43366). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cedf256 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cedd5b2 4. DB::ReadBuffer::next() @ 0x000000000b51c50b 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326cecab 6. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 11. asan_thread_start(void*) @ 0x000000000a7b5a5b 12. ? @ 0x00007f2ff11fbac3 13. ? @ 0x00007f2ff128d850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:05:13.005762 [ 1989 ] {62cf3c9b-af25-41dc-9f05-812e6b5ec4bc} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.450562312 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48792) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:13.070787 [ 1984 ] {ef7fc824-8274-4fba-b7e7-8b98a3d89345} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.449269799 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50354) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:13.605660 [ 1989 ] {62cf3c9b-af25-41dc-9f05-812e6b5ec4bc} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.450562312 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:13.613844 [ 1989 ] {62cf3c9b-af25-41dc-9f05-812e6b5ec4bc} TCPHandler: Client has gone away. 2024.07.22 12:05:13.652460 [ 1984 ] {ef7fc824-8274-4fba-b7e7-8b98a3d89345} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.449269799 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:13.656359 [ 1984 ] {ef7fc824-8274-4fba-b7e7-8b98a3d89345} TCPHandler: Client has gone away. 2024.07.22 12:05:14.167092 [ 2956 ] {ea81a92c-deb6-4908-84bc-ce36e217817c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.889622592 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:15.988833 [ 2956 ] {daed227f-716f-4cc2-80e1-6a3008547507} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.423205073 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:50864) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:16.196033 [ 1989 ] {968319ad-22e4-4e16-9aa4-bbcbeadbc5c0} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:48800). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48800) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001cee55fb 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001cee3907 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000326eff17 5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000326ef4ab 6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000326d069e 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:16.527027 [ 1989 ] {968319ad-22e4-4e16-9aa4-bbcbeadbc5c0} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:48800). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001cee55fb 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001cee3907 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x00000000327142cc 5. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x00000000326f19e8 6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000326d24c7 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:05:16.527498 [ 1989 ] {968319ad-22e4-4e16-9aa4-bbcbeadbc5c0} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:48800). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001cee55fb 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001cee3907 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000326eff17 5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000326ef4ab 6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000326d069e 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:16.532115 [ 1989 ] {968319ad-22e4-4e16-9aa4-bbcbeadbc5c0} TCPHandler: Client has gone away. 2024.07.22 12:05:16.579689 [ 2956 ] {daed227f-716f-4cc2-80e1-6a3008547507} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.423205073 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:19.504790 [ 1984 ] {619fd051-7c72-40a7-91de-3368919e895a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.047991904 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38858) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bf797 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:19.745399 [ 1985 ] {b32adc83-d73c-4e5f-a0a5-031d8a1c8cb4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.10799144 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49260) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bf797 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:20.520501 [ 1989 ] {90c950a8-5643-4986-aa0e-19c3194ae857} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.054934718 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60748) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:20.532101 [ 2956 ] {97827662-45e1-4b53-bf4f-115ed6dc5a71} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.58345403 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:52812) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:20.532112 [ 1987 ] {5f9f60b6-0226-46c6-b01b-933ea9805642} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.945931024 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54600) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:20.800342 [ 1987 ] {5f9f60b6-0226-46c6-b01b-933ea9805642} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.945931024 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:20.807378 [ 1987 ] {5f9f60b6-0226-46c6-b01b-933ea9805642} TCPHandler: Client has gone away. 2024.07.22 12:05:20.834316 [ 1985 ] {b32adc83-d73c-4e5f-a0a5-031d8a1c8cb4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.10799144 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bf797 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:20.843018 [ 1985 ] {b32adc83-d73c-4e5f-a0a5-031d8a1c8cb4} TCPHandler: Client has gone away. 2024.07.22 12:05:20.848571 [ 1989 ] {90c950a8-5643-4986-aa0e-19c3194ae857} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:60748). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001cee55fb 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001cee3582 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x00000000327142cc 5. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x00000000326f19e8 6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000326d24c7 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:05:20.849207 [ 1989 ] {90c950a8-5643-4986-aa0e-19c3194ae857} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.054934718 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:20.854541 [ 1989 ] {90c950a8-5643-4986-aa0e-19c3194ae857} TCPHandler: Client has gone away. 2024.07.22 12:05:20.880371 [ 1984 ] {619fd051-7c72-40a7-91de-3368919e895a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.047991904 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bf797 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:20.881888 [ 1984 ] {619fd051-7c72-40a7-91de-3368919e895a} TCPHandler: Client has gone away. 2024.07.22 12:05:21.059733 [ 2956 ] {97827662-45e1-4b53-bf4f-115ed6dc5a71} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.58345403 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.207164 [ 1989 ] {509d6f27-575b-4694-97ae-0bdacf2d12d4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.290061627 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60760) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003282918f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 9. asan_thread_start(void*) @ 0x000000000a7b5a5b 10. ? @ 0x00007f2ff11fbac3 11. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.220422 [ 1989 ] {509d6f27-575b-4694-97ae-0bdacf2d12d4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.290061627 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003282918f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 9. asan_thread_start(void*) @ 0x000000000a7b5a5b 10. ? @ 0x00007f2ff11fbac3 11. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.322183 [ 1984 ] {3bb68080-43c0-4134-9d24-f2b013c1056c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.131991253 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38872) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bf797 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.383171 [ 1985 ] {e60dbe7d-46e7-42f6-b571-68a111a7d531} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.179533478 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49266) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003282918f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 9. asan_thread_start(void*) @ 0x000000000a7b5a5b 10. ? @ 0x00007f2ff11fbac3 11. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.392160 [ 1985 ] {e60dbe7d-46e7-42f6-b571-68a111a7d531} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.179533478 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003282918f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 9. asan_thread_start(void*) @ 0x000000000a7b5a5b 10. ? @ 0x00007f2ff11fbac3 11. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.397381 [ 2956 ] {e6c9d1ac-260f-45ce-a846-cd197cf50e7a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.984006152 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:52824) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.440494 [ 1985 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:49266). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cedf256 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cedd5b2 4. DB::ReadBuffer::next() @ 0x000000000b51c50b 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326cecab 6. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 11. asan_thread_start(void*) @ 0x000000000a7b5a5b 12. ? @ 0x00007f2ff11fbac3 13. ? @ 0x00007f2ff128d850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:05:23.471427 [ 1989 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:60760). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000001cedf256 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001cedd5b2 4. DB::ReadBuffer::next() @ 0x000000000b51c50b 5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x00000000326cecab 6. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000032714671 7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003a1d22ef 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003a1d2f97 9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003a5dff1c 10. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003a5d8f08 11. asan_thread_start(void*) @ 0x000000000a7b5a5b 12. ? @ 0x00007f2ff11fbac3 13. ? @ 0x00007f2ff128d850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 12:05:23.505021 [ 1987 ] {0bc8f585-06b3-471d-bbe1-a0f8615b04a8} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.445036816 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54610) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.823023 [ 1984 ] {3bb68080-43c0-4134-9d24-f2b013c1056c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.131991253 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002b3bf797 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003284c719 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003282d1a3 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003282e3c6 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd4bcbb 9. ./src/Common/ThreadPool.h:224: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001cd54853 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 12. asan_thread_start(void*) @ 0x000000000a7b5a5b 13. ? @ 0x00007f2ff11fbac3 14. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.826746 [ 1984 ] {3bb68080-43c0-4134-9d24-f2b013c1056c} TCPHandler: Client has gone away. 2024.07.22 12:05:23.931586 [ 2956 ] {e6c9d1ac-260f-45ce-a846-cd197cf50e7a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.984006152 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.948036 [ 1987 ] {0bc8f585-06b3-471d-bbe1-a0f8615b04a8} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.445036816 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003a4c9f53 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001cb401b0 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002b3c307d 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002b3c207f 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000002e8313d1 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000032829c35 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000328292c1 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000032856445 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001cd43e8d 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001cd4ff4d 10. asan_thread_start(void*) @ 0x000000000a7b5a5b 11. ? @ 0x00007f2ff11fbac3 12. ? @ 0x00007f2ff128d850 2024.07.22 12:05:23.950931 [ 1987 ] {0bc8f585-06b3-471d-bbe1-a0f8615b04a8} TCPHandler: Client has gone away. 2024.07.22 12:08:50.962986 [ 390 ] {} Application: Closed all listening sockets. Waiting for 2 outstanding connections.