2024.07.22 10:03:44.861362 [ 387 ] {} Context: Server was built with sanitizer. It will work slowly. 2024.07.22 10:03:44.881916 [ 387 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2024.07.22 10:03:44.883230 [ 387 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.07.22 10:03:54.608364 [ 387 ] {} CloudPlacementInfo: Placement info has not been loaded 2024.07.22 10:03:54.786646 [ 387 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2024.07.22 10:03:54.787566 [ 387 ] {} KeeperLogStore: Removing all changelogs 2024.07.22 10:03:54.788845 [ 387 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.07.22 10:03:54.793213 [ 387 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.07.22 10:03:54.793263 [ 387 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.07.22 10:03:54.844210 [ 1095 ] {} RaftInstance: Election timeout, initiate leader election 2024.07.22 10:03:54.919488 [ 387 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2024.07.22 10:03:54.919564 [ 387 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2024.07.22 10:03:55.254131 [ 387 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.07.22 10:03:56.885693 [ 390 ] {19ff783c-6031-4993-859a-abec3fdd0aee} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.07.22 10:04:05.381258 [ 390 ] {f697446e-fd66-4580-83ee-dad52b2d560e} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.07.22 10:09:04.942893 [ 1745 ] {3a3449f7-21f7-4e40-828c-5f8884abd500} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:51292) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003190248c 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6d2f84 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5ee643 7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b64c611 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000357100cd 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003570fae1 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034853e1d 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000034796ba3 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347904a8 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5546: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347de611 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000034793181 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347904a8 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347ae283 17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034771d0e 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000034768050 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x0000000034756264 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000003570e2c0 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000035705385 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000035711355 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035710ab3 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000355727a4 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000036565b80 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003655a2e3 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4c7508 28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 2024.07.22 10:09:04.944190 [ 1745 ] {3a3449f7-21f7-4e40-828c-5f8884abd500} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003190248c 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6d2f84 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5ee643 7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b64c611 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000357100cd 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003570fae1 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034853e1d 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000034796ba3 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347904a8 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5546: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347de611 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000034793181 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347904a8 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347ae283 17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034771d0e 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000034768050 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x0000000034756264 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000003570e2c0 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000035705385 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000035711355 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035710ab3 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000355727a4 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000036565b80 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003655a2e3 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4c7508 28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 2024.07.22 10:09:05.313390 [ 1745 ] {7505061e-cf7e-4c43-876c-1579c7ae0540} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:51292) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003190248c 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6d2f84 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5ee643 7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b64c611 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000357100cd 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003570fae1 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000036568025 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003655a2e3 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4c7508 13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eadaa8e 18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003ead7972 19. ? @ 0x00007fb033d1dac3 20. ? @ 0x00007fb033daf850 2024.07.22 10:09:05.314075 [ 1745 ] {7505061e-cf7e-4c43-876c-1579c7ae0540} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003190248c 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6d2f84 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5ee643 7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b64c611 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000357100cd 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003570fae1 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000036568025 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003655a2e3 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4c7508 13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eadaa8e 18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003ead7972 19. ? @ 0x00007fb033d1dac3 20. ? @ 0x00007fb033daf850 2024.07.22 10:09:07.397682 [ 1745 ] {744142c6-ae7f-4cdf-99bb-7f19ea0827c0} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 18.00 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:51292) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:09:07.591295 [ 1745 ] {744142c6-ae7f-4cdf-99bb-7f19ea0827c0} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 18.00 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:09:08.628360 [ 1745 ] {f8134cdf-333a-4a58-a995-4bbf187f79ca} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:51292) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003190248c 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6d2f84 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5ee643 7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b64c611 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000357100cd 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003570fae1 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034853e1d 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000034796ba3 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347904a8 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5546: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347de611 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000034793181 15. ./contrib/llvm-project/libcxx/include/vector:434: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034772481 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000034768050 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x0000000034756264 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000003570e2c0 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000035705385 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000035711355 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035710ab3 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000355727a4 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000036565b80 24. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003655a2e3 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4c7508 26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 30. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eadaa8e 31. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003ead7972 2024.07.22 10:09:08.629257 [ 1745 ] {f8134cdf-333a-4a58-a995-4bbf187f79ca} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003190248c 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6d2f84 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5ee643 7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b64c611 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000357100cd 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003570fae1 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034853e1d 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000034796ba3 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347904a8 13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5546: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347de611 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000034793181 15. ./contrib/llvm-project/libcxx/include/vector:434: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034772481 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000034768050 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x0000000034756264 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000003570e2c0 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000035705385 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000035711355 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035710ab3 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000355727a4 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000036565b80 24. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003655a2e3 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4c7508 26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 30. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eadaa8e 31. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003ead7972 2024.07.22 10:09:09.977951 [ 1745 ] {d22f4b02-1d10-470c-890f-0a10f39d81db} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.52 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:51292) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:09:09.993668 [ 1745 ] {d22f4b02-1d10-470c-890f-0a10f39d81db} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.52 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000031902902 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031902035 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:10:40.080799 [ 1115 ] {} CgroupsMemoryUsageObserver: Exceeded sort memory limit (24.79 GiB) 2024.07.22 10:16:16.308315 [ 1990 ] {aee68474-94fd-492e-bcdf-bca2a99b26da} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.157630881 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:40942) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:16.833264 [ 1990 ] {aee68474-94fd-492e-bcdf-bca2a99b26da} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.157630881 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:18.365302 [ 1990 ] {d9228c82-f395-4ad5-b408-f8987a31a296} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020773137 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:37938) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:18.592483 [ 1990 ] {d9228c82-f395-4ad5-b408-f8987a31a296} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020773137 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:20.223611 [ 1990 ] {e2233228-4349-4031-bb84-9d07f671dd9c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.09270127 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:37950) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:20.874397 [ 1990 ] {e2233228-4349-4031-bb84-9d07f671dd9c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.09270127 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:21.448483 [ 3040 ] {227f5223-b79f-4e6e-aea1-37c6db561da1} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 580.54 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:40940) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031902e97 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003190229c 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:16:21.500376 [ 3040 ] {227f5223-b79f-4e6e-aea1-37c6db561da1} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 580.54 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031902e97 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003190229c 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:16:22.564112 [ 1990 ] {48e82893-ff11-4291-be69-ef2014d8d481} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.014877468 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:37952) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:103: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a6586b6 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 7. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 9. ? @ 0x00007fb033d1dac3 10. ? @ 0x00007fb033daf850 2024.07.22 10:16:22.619312 [ 1990 ] {48e82893-ff11-4291-be69-ef2014d8d481} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.014877468 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:103: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a6586b6 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 7. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 9. ? @ 0x00007fb033d1dac3 10. ? @ 0x00007fb033daf850 2024.07.22 10:16:23.774605 [ 3040 ] {a4335a6b-1835-40d7-b911-95084017b384} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 580.17 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:40940) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031902e97 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003190229c 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:16:23.818812 [ 3040 ] {a4335a6b-1835-40d7-b911-95084017b384} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 580.17 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031902e97 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003190229c 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:16:24.446397 [ 1990 ] {4e530c44-9637-4e2a-a22c-0f8380b804b0} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.08779138 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:37958) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:25.157789 [ 1990 ] {4e530c44-9637-4e2a-a22c-0f8380b804b0} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.08779138 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:26.399355 [ 3040 ] {6e9bd87f-6db3-4c2e-b716-d0a38a1540c2} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 576.46 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:40940) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031902e97 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003190229c 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:16:26.736124 [ 1990 ] {2da7237f-d881-43fe-83e5-d372a6435ad9} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.025709471 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:37972) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:26.747115 [ 3040 ] {6e9bd87f-6db3-4c2e-b716-d0a38a1540c2} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 576.46 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031902e97 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003190229c 4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x00000000318a50e2 5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a68d5ce 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a65ca67 7. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003a65dca6 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000001f33290f 9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f33d373 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f33d21f 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 2024.07.22 10:16:26.932874 [ 1990 ] {2da7237f-d881-43fe-83e5-d372a6435ad9} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.025709471 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:29.501500 [ 1990 ] {58ff8895-ddaf-4957-a183-cd1e2b104ccd} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.072435213 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:45002) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:29.589417 [ 1988 ] {99f98049-abf8-49d5-9e01-c078ea828688} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.573735259 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57060) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:29.724349 [ 3029 ] {eab07a38-9b5c-4684-895a-70dce9fb7211} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.59471179 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:41038) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:30.022764 [ 1988 ] {99f98049-abf8-49d5-9e01-c078ea828688} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.573735259 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:30.045238 [ 1988 ] {99f98049-abf8-49d5-9e01-c078ea828688} TCPHandler: Client has gone away. 2024.07.22 10:16:30.101681 [ 3029 ] {eab07a38-9b5c-4684-895a-70dce9fb7211} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.59471179 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:30.113307 [ 3029 ] {eab07a38-9b5c-4684-895a-70dce9fb7211} TCPHandler: Client has gone away. 2024.07.22 10:16:30.338802 [ 1989 ] {4e9fe8c9-201a-4293-8258-420f993f944e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.389078191 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47948) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:30.662985 [ 3037 ] {9c3001f4-b5ab-42cc-ac83-79fa16ea4473} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.168190894 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55254) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:30.713347 [ 3032 ] {f95f956c-5250-4d2c-a819-12afa3143b81} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.304067202 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51604) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:30.804683 [ 3033 ] {1904797d-e4e3-4f57-a582-eeae08db53de} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.565042462 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33684) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:30.818422 [ 3034 ] {9b69686a-ec69-4914-acca-c12bf08d134d} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.453500457 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56528) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:31.346251 [ 1990 ] {58ff8895-ddaf-4957-a183-cd1e2b104ccd} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.072435213 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:31.346338 [ 3037 ] {9c3001f4-b5ab-42cc-ac83-79fa16ea4473} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:55254). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001f4f235f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f4f0cd3 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a5162fe 5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4ee30b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4c8101 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eadaa8e 12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003ead7972 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 10:16:31.349451 [ 3037 ] {9c3001f4-b5ab-42cc-ac83-79fa16ea4473} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.168190894 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:31.357995 [ 3037 ] {9c3001f4-b5ab-42cc-ac83-79fa16ea4473} TCPHandler: Client has gone away. 2024.07.22 10:16:32.259054 [ 3033 ] {1904797d-e4e3-4f57-a582-eeae08db53de} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:33684). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001f4f235f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f4f0739 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a5162fe 5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4ee30b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4c8101 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eadaa8e 12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003ead7972 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 10:16:32.259100 [ 3034 ] {9b69686a-ec69-4914-acca-c12bf08d134d} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:56528). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001f4f235f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f4f0cd3 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a5162fe 5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4ee30b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4c8101 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eadaa8e 12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003ead7972 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 10:16:32.259257 [ 3033 ] {1904797d-e4e3-4f57-a582-eeae08db53de} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.565042462 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:32.259290 [ 3034 ] {9b69686a-ec69-4914-acca-c12bf08d134d} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.453500457 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:32.265329 [ 3034 ] {9b69686a-ec69-4914-acca-c12bf08d134d} TCPHandler: Client has gone away. 2024.07.22 10:16:32.265368 [ 3033 ] {1904797d-e4e3-4f57-a582-eeae08db53de} TCPHandler: Client has gone away. 2024.07.22 10:16:33.030207 [ 3037 ] {7ab6f44f-985c-4ac8-85d4-41951624b8d8} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.086774898 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:45018) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:33.437079 [ 3037 ] {7ab6f44f-985c-4ac8-85d4-41951624b8d8} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.086774898 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:33.673379 [ 1989 ] {4e9fe8c9-201a-4293-8258-420f993f944e} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:47948). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001f4f235f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f4f0739 4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a5162fe 5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4ee30b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4c8101 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a51679c 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e655a00 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e656881 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eaddcc6 11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eadaa8e 12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003ead7972 13. ? @ 0x00007fb033d1dac3 14. ? @ 0x00007fb033daf850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 10:16:33.673629 [ 1989 ] {4e9fe8c9-201a-4293-8258-420f993f944e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.389078191 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:33.674288 [ 1989 ] {4e9fe8c9-201a-4293-8258-420f993f944e} TCPHandler: Client has gone away. 2024.07.22 10:16:33.707426 [ 3032 ] {f95f956c-5250-4d2c-a819-12afa3143b81} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.304067202 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:33.708319 [ 3032 ] {f95f956c-5250-4d2c-a819-12afa3143b81} TCPHandler: Client has gone away. 2024.07.22 10:16:35.014926 [ 3037 ] {9436909e-4b6f-4212-94b0-e06ce221ef1e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.077251109 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:45032) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:16:35.733276 [ 3037 ] {9436909e-4b6f-4212-94b0-e06ce221ef1e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.077251109 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9a7a09 1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f0fb1b3 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x00000000318a850c 3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318a76c9 4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035a8009b 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a658fb1 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a65886a 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a69889d 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000001f32a98b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000001f33736b 10. ? @ 0x00007fb033d1dac3 11. ? @ 0x00007fb033daf850 2024.07.22 10:25:13.593888 [ 387 ] {} Application: Closed all listening sockets. Waiting for 4 outstanding connections. 2024.07.22 10:25:20.121835 [ 1096 ] {} RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled