2024.08.31 19:49:48.522749 [ 391 ] {} Context: Server was built with sanitizer. It will work slowly. 2024.08.31 19:49:48.532341 [ 391 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2024.08.31 19:49:48.532630 [ 391 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.08.31 19:49:48.861571 [ 391 ] {} CloudPlacementInfo: Placement info has not been loaded 2024.08.31 19:49:48.917473 [ 391 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2024.08.31 19:49:48.917681 [ 391 ] {} KeeperLogStore: Removing all changelogs 2024.08.31 19:49:48.918053 [ 391 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.08.31 19:49:48.918066 [ 391 ] {} KeeperServer: Can't start NON-SECURE keeper server in FIPS mode, please check the config. 2024.08.31 19:49:48.918864 [ 391 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.08.31 19:49:48.918874 [ 391 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.08.31 19:49:48.969246 [ 970 ] {} RaftInstance: Election timeout, initiate leader election 2024.08.31 19:49:48.982138 [ 391 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2024.08.31 19:49:48.982161 [ 391 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2024.08.31 19:49:49.023640 [ 391 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.08.31 19:49:50.487568 [ 394 ] {1352c03c-b3ff-420d-a719-0d681e0b4b97} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.08.31 19:49:58.924624 [ 394 ] {2af38560-b531-4185-9324-42b9a0a4037e} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.08.31 19:51:33.744804 [ 1767 ] {27c6a68e-446d-42af-a41b-c8c4197616b0} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:32878) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ef7d6c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e8d5c5 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ebeabd 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fa957e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fa901c 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2109: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030887eb7 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003082a63c 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030828c41 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003084cbdd 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003082a0c3 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030828c41 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003083802c 17. ./contrib/llvm-project/libcxx/include/vector:951: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000308160aa 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x00000000308126dd 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000003080b226 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000030fa81f6 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000030fa5303 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000030faab58 23. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000030fa9d68 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000030ec7463 25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003169c9b9 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031695ea7 27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000334396c6 28. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 2024.08.31 19:51:33.745010 [ 1767 ] {27c6a68e-446d-42af-a41b-c8c4197616b0} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ef7d6c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e8d5c5 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ebeabd 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fa957e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fa901c 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2109: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030887eb7 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003082a63c 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030828c41 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003084cbdd 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003082a0c3 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030828c41 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003083802c 17. ./contrib/llvm-project/libcxx/include/vector:951: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000308160aa 18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x00000000308126dd 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000003080b226 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000030fa81f6 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000030fa5303 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000030faab58 23. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000030fa9d68 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000030ec7463 25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003169c9b9 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031695ea7 27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000334396c6 28. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 2024.08.31 19:51:33.766633 [ 1767 ] {78157804-e812-4440-beb7-f7c251e91757} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:32878) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ef7d6c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e8d5c5 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ebeabd 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fa957e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fa901c 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003169d7fe 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031695ea7 12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000334396c6 13. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 17. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 18. ? @ 0x00007fadc2097ac3 19. ? @ 0x00007fadc2129850 2024.08.31 19:51:33.766817 [ 1767 ] {78157804-e812-4440-beb7-f7c251e91757} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ef7d6c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e8d5c5 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ebeabd 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fa957e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fa901c 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003169d7fe 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031695ea7 12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000334396c6 13. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 17. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 18. ? @ 0x00007fadc2097ac3 19. ? @ 0x00007fadc2129850 2024.08.31 19:51:33.842047 [ 1767 ] {0728b04f-2b6d-4f8a-981e-431ed7fb10c4} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 16.15 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:32878) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:51:33.845114 [ 1767 ] {0728b04f-2b6d-4f8a-981e-431ed7fb10c4} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 16.15 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:51:33.875797 [ 1767 ] {6dd00616-ec1e-4d79-a838-281aa2ca9154} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:32878) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ef7d6c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e8d5c5 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ebeabd 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fa957e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fa901c 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2109: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030887eb7 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003082a63c 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030828c41 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003084cbdd 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003082a0c3 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000308164e1 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x00000000308126dd 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000003080b226 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000030fa81f6 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000030fa5303 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000030faab58 21. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000030fa9d68 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000030ec7463 23. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003169c9b9 24. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031695ea7 25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000334396c6 26. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 30. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 31. ? @ 0x00007fadc2097ac3 2024.08.31 19:51:33.875978 [ 1767 ] {6dd00616-ec1e-4d79-a838-281aa2ca9154} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ef7d6c 6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000033e8d5c5 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000033ebeabd 8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000030fa957e 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000030fa901c 10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2109: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000030887eb7 11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003082a63c 12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6624: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000030828c41 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003084cbdd 14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003082a0c3 15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000308164e1 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x00000000308126dd 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000003080b226 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000030fa81f6 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000030fa5303 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000030faab58 21. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:40: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000030fa9d68 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000030ec7463 23. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003169c9b9 24. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000031695ea7 25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000334396c6 26. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 30. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 31. ? @ 0x00007fadc2097ac3 2024.08.31 19:51:33.895926 [ 1767 ] {187f8d18-569e-45ca-a45b-3d822c6eac53} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:32878) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:51:33.896595 [ 1767 ] {187f8d18-569e-45ca-a45b-3d822c6eac53} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000002f3f90f1 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8e49 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:52:56.087235 [ 2039 ] {b388ae8e-3d57-4033-b9e6-13783a8c77ad} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:54426) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f3f9392 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8f22 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003350b2d7 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350aeea 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:52:56.087660 [ 2039 ] {b388ae8e-3d57-4033-b9e6-13783a8c77ad} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f3f9392 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8f22 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003350b2d7 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350aeea 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:52:56.730267 [ 2039 ] {d386db42-acbd-4023-8eae-bb121e55437b} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:54426) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f3f9392 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8f22 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003350b2d7 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350aeea 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:52:56.730709 [ 2039 ] {d386db42-acbd-4023-8eae-bb121e55437b} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f3f9392 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8f22 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003350b2d7 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350aeea 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:52:57.345388 [ 2039 ] {881cf730-5978-497e-b433-618684d7acea} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:54426) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f3f9392 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8f22 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003350b2d7 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350aeea 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:52:57.345747 [ 2039 ] {881cf730-5978-497e-b433-618684d7acea} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.47 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002f3f9392 4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002f3f8f22 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d259d 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003350b2d7 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350aeea 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:52:58.685857 [ 2039 ] {7537a40f-80ff-4cd2-8684-e81b17c1d186} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.155910116 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:33108) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:52:58.704933 [ 2039 ] {7537a40f-80ff-4cd2-8684-e81b17c1d186} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.155910116 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:52:58.908195 [ 2041 ] {c1c79362-609b-4785-9cea-361d87dc9310} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.103351986 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:43536) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:52:58.960058 [ 2041 ] {c1c79362-609b-4785-9cea-361d87dc9310} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:43536). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002482f8c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002482e60f 5. DB::WriteBuffer::next() @ 0x000000001802d58c 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x0000000033459fe3 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000033439ff9 8. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:52:58.960180 [ 2041 ] {c1c79362-609b-4785-9cea-361d87dc9310} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.103351986 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:52:58.975246 [ 2041 ] {c1c79362-609b-4785-9cea-361d87dc9310} TCPHandler: Client has gone away. 2024.08.31 19:53:00.285593 [ 2041 ] {bb4305b1-c2f6-42c8-ac4a-1d985de00bcc} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.052165122 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:43544) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:00.302142 [ 2041 ] {bb4305b1-c2f6-42c8-ac4a-1d985de00bcc} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.052165122 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:00.531114 [ 2039 ] {ff819880-d584-4dbc-a2af-158e9a56c515} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.61152871 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:33128) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:00.545179 [ 2041 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:43544). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:00.545209 [ 2042 ] {dc99b213-eca3-4195-ad2a-dad154212760} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.417194432 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:59688) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:00.549884 [ 2039 ] {ff819880-d584-4dbc-a2af-158e9a56c515} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.61152871 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:00.570776 [ 2042 ] {dc99b213-eca3-4195-ad2a-dad154212760} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.417194432 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:00.571108 [ 2042 ] {dc99b213-eca3-4195-ad2a-dad154212760} TCPHandler: Client has gone away. 2024.08.31 19:53:02.542781 [ 2041 ] {de1c8a20-2ed4-41c2-8b78-876234fc4514} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.544602698 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:59690) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:02.560889 [ 2041 ] {de1c8a20-2ed4-41c2-8b78-876234fc4514} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.544602698 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:02.565038 [ 2039 ] {b0719953-f1cc-4821-aa1f-18345bef4574} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.799987942 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:33130) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:02.566463 [ 2041 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:59690). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:02.583738 [ 2039 ] {b0719953-f1cc-4821-aa1f-18345bef4574} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.799987942 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:02.604997 [ 2042 ] {83f7817d-326e-4e20-9bb2-13e790d3a35a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.603628021 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:46320) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:02.605070 [ 2040 ] {d9084206-f7bd-4849-834a-9ade28a1db42} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.620491864 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:43546) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:02.622925 [ 2042 ] {83f7817d-326e-4e20-9bb2-13e790d3a35a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.603628021 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:02.623215 [ 2042 ] {83f7817d-326e-4e20-9bb2-13e790d3a35a} TCPHandler: Client has gone away. 2024.08.31 19:53:02.626739 [ 2040 ] {d9084206-f7bd-4849-834a-9ade28a1db42} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.620491864 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:02.626930 [ 2040 ] {d9084206-f7bd-4849-834a-9ade28a1db42} TCPHandler: Client has gone away. 2024.08.31 19:53:04.433202 [ 2039 ] {2e7d607e-78b6-454a-ab15-61309036aee5} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.66620202 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:33172) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:04.434715 [ 2041 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:52716). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:04.455412 [ 2039 ] {2e7d607e-78b6-454a-ab15-61309036aee5} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.66620202 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:04.487333 [ 2040 ] {f1c016d2-8ccf-4a5b-a5f7-170faf638a4e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.489708655 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:46324) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:04.505416 [ 2040 ] {f1c016d2-8ccf-4a5b-a5f7-170faf638a4e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.489708655 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:04.505679 [ 2040 ] {f1c016d2-8ccf-4a5b-a5f7-170faf638a4e} TCPHandler: Client has gone away. 2024.08.31 19:53:05.881575 [ 2039 ] {a40bff8d-c975-474a-b925-dc58bd22bc4d} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.02833139 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:43572) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:05.883453 [ 1767 ] {539a6069-d91b-48fa-9fb8-411cd0c7be1f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.025495812 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:59704) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:05.926877 [ 1767 ] {539a6069-d91b-48fa-9fb8-411cd0c7be1f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.025495812 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:05.947955 [ 2039 ] {a40bff8d-c975-474a-b925-dc58bd22bc4d} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.02833139 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:05.986782 [ 2041 ] {0e7c7be5-4c02-48d4-b38b-54aa23f7108c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.120012071 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:52726) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.017444 [ 2041 ] {0e7c7be5-4c02-48d4-b38b-54aa23f7108c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.120012071 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.116896 [ 2042 ] {163a38d6-f7a8-4df5-8232-ffbe1ec8d9f6} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.138394337 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:43560) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.137772 [ 1613 ] {a587369c-de1b-48f7-bf46-e09779642b03} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.144448423 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:59702) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.143199 [ 2042 ] {163a38d6-f7a8-4df5-8232-ffbe1ec8d9f6} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:43560). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002482f8c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002482e60f 5. DB::WriteBuffer::next() @ 0x000000001802d58c 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x0000000033459fe3 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000033439ff9 8. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:06.143325 [ 2042 ] {163a38d6-f7a8-4df5-8232-ffbe1ec8d9f6} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.138394337 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.143492 [ 2042 ] {163a38d6-f7a8-4df5-8232-ffbe1ec8d9f6} TCPHandler: Client has gone away. 2024.08.31 19:53:06.160113 [ 1613 ] {a587369c-de1b-48f7-bf46-e09779642b03} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 3.144448423 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.160325 [ 1613 ] {a587369c-de1b-48f7-bf46-e09779642b03} TCPHandler: Client has gone away. 2024.08.31 19:53:06.538245 [ 2040 ] {a871874a-3cdd-48ad-9a31-a4b6cbf22f1a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004006831 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:33192) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d225a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.539697 [ 2035 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:60602). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:06.540312 [ 2041 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:52726). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:06.540845 [ 1767 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:59704). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:06.541208 [ 2039 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:43572). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:06.563727 [ 2040 ] {a871874a-3cdd-48ad-9a31-a4b6cbf22f1a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004006831 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d225a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.594081 [ 1736 ] {c32fcdcc-82e7-44c3-a5dd-be3dc6d5d35a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.730265429 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:46338) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.614016 [ 1736 ] {c32fcdcc-82e7-44c3-a5dd-be3dc6d5d35a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.730265429 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:06.614297 [ 1736 ] {c32fcdcc-82e7-44c3-a5dd-be3dc6d5d35a} TCPHandler: Client has gone away. 2024.08.31 19:53:08.164456 [ 2041 ] {6b98299b-c0d1-4778-9994-65d3d76c3d3e} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.288078507 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:46598) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.181676 [ 1613 ] {f62abb95-992f-4eff-b7a2-c2bca86ac1e1} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.305068759 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:41744) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.218536 [ 2041 ] {6b98299b-c0d1-4778-9994-65d3d76c3d3e} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.288078507 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.223655 [ 2042 ] {983953a8-56f9-49a8-be19-c94291d5fea4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.371706721 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:37884) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.226341 [ 2039 ] {a63b7944-b135-49d2-953d-c78939b9d5db} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.471762776 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:59370) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.231123 [ 2041 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:46598). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:08.234631 [ 1736 ] {f1abde8e-e61c-4ba3-9206-b72a050f5296} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.31816014 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:39314) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.240786 [ 2040 ] {129615ba-5fe8-4e0e-af3c-78237231c2f5} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.332171113 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:54358) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.243715 [ 1613 ] {f62abb95-992f-4eff-b7a2-c2bca86ac1e1} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.305068759 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.244007 [ 1613 ] {f62abb95-992f-4eff-b7a2-c2bca86ac1e1} TCPHandler: Client has gone away. 2024.08.31 19:53:08.257936 [ 1736 ] {f1abde8e-e61c-4ba3-9206-b72a050f5296} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.31816014 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.258274 [ 1736 ] {f1abde8e-e61c-4ba3-9206-b72a050f5296} TCPHandler: Client has gone away. 2024.08.31 19:53:08.264942 [ 2042 ] {983953a8-56f9-49a8-be19-c94291d5fea4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.371706721 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.269590 [ 2042 ] {983953a8-56f9-49a8-be19-c94291d5fea4} TCPHandler: Client has gone away. 2024.08.31 19:53:08.283854 [ 1767 ] {291bd7cd-2b2d-440e-bc5d-a05c057e0248} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:45798). (NETWORK_ERROR) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:45798) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002482f8c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002482ea7f 5. DB::WriteBuffer::next() @ 0x000000001802d58c 6. ./build_docker/./src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x0000000033458d3b 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000334585fd 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033439b9d 9. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 14. ? @ 0x00007fadc2097ac3 15. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.283943 [ 2040 ] {129615ba-5fe8-4e0e-af3c-78237231c2f5} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:54358). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002482f8c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002482e60f 5. DB::WriteBuffer::next() @ 0x000000001802d58c 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x0000000033459fe3 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000033439ff9 8. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:08.284017 [ 2040 ] {129615ba-5fe8-4e0e-af3c-78237231c2f5} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.332171113 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.284204 [ 2040 ] {129615ba-5fe8-4e0e-af3c-78237231c2f5} TCPHandler: Client has gone away. 2024.08.31 19:53:08.290076 [ 2039 ] {a63b7944-b135-49d2-953d-c78939b9d5db} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.471762776 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.326765 [ 1767 ] {291bd7cd-2b2d-440e-bc5d-a05c057e0248} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:45798). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002482f8c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002482ea7f 5. DB::WriteBuffer::next() @ 0x000000001802d58c 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x0000000033459fe3 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000033439ff9 8. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:08.326871 [ 1767 ] {291bd7cd-2b2d-440e-bc5d-a05c057e0248} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:45798). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002482f8c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002482ea7f 5. DB::WriteBuffer::next() @ 0x000000001802d58c 6. ./build_docker/./src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x0000000033458d3b 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000334585fd 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033439b9d 9. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 14. ? @ 0x00007fadc2097ac3 15. ? @ 0x00007fadc2129850 2024.08.31 19:53:08.327092 [ 1767 ] {291bd7cd-2b2d-440e-bc5d-a05c057e0248} TCPHandler: Client has gone away. 2024.08.31 19:53:09.796617 [ 1767 ] {cee9f94c-1b3f-4576-8d2f-251df43c4475} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.115185937 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:37900) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:09.835646 [ 1736 ] {70e256a2-ce00-49f3-b38a-1ecf4c9956ec} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116008707 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:54368) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:09.933940 [ 1767 ] {cee9f94c-1b3f-4576-8d2f-251df43c4475} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.115185937 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:09.993195 [ 1736 ] {70e256a2-ce00-49f3-b38a-1ecf4c9956ec} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116008707 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.031264 [ 2042 ] {d00d5054-477e-4847-b178-88c9ec646d67} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.371194071 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:46604) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.032891 [ 2035 ] {2c837cc9-3ce8-4f14-981f-0e9dfc80b3af} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.184088122 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:45814) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.037443 [ 2037 ] {1ee884c6-1a08-450a-8772-9f5ecbcd6854} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.322995277 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:41746) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.038323 [ 1613 ] {1d4679cb-8736-4335-a8df-942e049e77c8} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.164107673 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:41590) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.053638 [ 2035 ] {2c837cc9-3ce8-4f14-981f-0e9dfc80b3af} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.184088122 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.058819 [ 1613 ] {1d4679cb-8736-4335-a8df-942e049e77c8} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.164107673 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.058883 [ 2042 ] {d00d5054-477e-4847-b178-88c9ec646d67} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.371194071 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.060718 [ 2037 ] {1ee884c6-1a08-450a-8772-9f5ecbcd6854} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.322995277 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.725648 [ 2041 ] {5b9d8f63-4c89-4064-a158-a988b6166ea2} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.264966824 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:59374) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.727354 [ 1613 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:41590). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:10.727522 [ 2035 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:45814). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:10.727740 [ 2037 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:41746). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:10.727914 [ 1736 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54368). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:10.728078 [ 1767 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:37900). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:10.728223 [ 2042 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:46604). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:10.743486 [ 2036 ] {8e662cfb-4fbf-4561-b8b0-cb2473c7c076} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.002164992 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:39324) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.776293 [ 2041 ] {5b9d8f63-4c89-4064-a158-a988b6166ea2} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.264966824 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.784527 [ 2036 ] {8e662cfb-4fbf-4561-b8b0-cb2473c7c076} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.002164992 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:10.784854 [ 2036 ] {8e662cfb-4fbf-4561-b8b0-cb2473c7c076} TCPHandler: Client has gone away. 2024.08.31 19:53:12.216309 [ 1736 ] {b639c037-683c-4c74-b68e-1c2064ea9c6b} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.119294877 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:54380) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.221250 [ 2035 ] {8c044883-8bb8-4ce4-8170-7bd36a8840af} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.067683681 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:41758) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.234623 [ 2042 ] {23873605-39fb-4561-97dc-05a7b2017485} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.096563461 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:46614) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.237774 [ 1736 ] {b639c037-683c-4c74-b68e-1c2064ea9c6b} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.119294877 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.240070 [ 2035 ] {8c044883-8bb8-4ce4-8170-7bd36a8840af} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.067683681 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.301712 [ 2042 ] {23873605-39fb-4561-97dc-05a7b2017485} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.096563461 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.343893 [ 1613 ] {e3f7e9e6-b397-458c-8801-b7f1ffefdd2d} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.254001234 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:39328) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.344104 [ 2041 ] {3e836d9c-b3bf-4ce1-9cc5-ac1085f1aef1} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.365062955 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:59386) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.345811 [ 2035 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:41758). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:12.360240 [ 1736 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54380). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:12.363284 [ 2042 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:46614). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:12.400375 [ 2036 ] {cf523255-bb2f-45e6-89f0-7063d357278c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.202908963 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:45818) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.400930 [ 1613 ] {e3f7e9e6-b397-458c-8801-b7f1ffefdd2d} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.254001234 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.407938 [ 1613 ] {e3f7e9e6-b397-458c-8801-b7f1ffefdd2d} TCPHandler: Client has gone away. 2024.08.31 19:53:12.413027 [ 1767 ] {6b2f9428-3781-409c-870f-c8a7f1ceb2cf} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.265342582 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:37910) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.417073 [ 2037 ] {9b2ff780-0e00-4c26-bf16-2476a0da4fea} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:41604). (NETWORK_ERROR) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:41604) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002482f8c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002482ea7f 5. DB::WriteBuffer::next() @ 0x000000001802d58c 6. ./build_docker/./src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x0000000033458d3b 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000334585fd 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033439b9d 9. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 14. ? @ 0x00007fadc2097ac3 15. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.427638 [ 2036 ] {cf523255-bb2f-45e6-89f0-7063d357278c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.202908963 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.427877 [ 2036 ] {cf523255-bb2f-45e6-89f0-7063d357278c} TCPHandler: Client has gone away. 2024.08.31 19:53:12.428927 [ 2041 ] {3e836d9c-b3bf-4ce1-9cc5-ac1085f1aef1} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.365062955 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.431214 [ 1767 ] {6b2f9428-3781-409c-870f-c8a7f1ceb2cf} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.265342582 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.431386 [ 1767 ] {6b2f9428-3781-409c-870f-c8a7f1ceb2cf} TCPHandler: Client has gone away. 2024.08.31 19:53:12.434076 [ 2037 ] {9b2ff780-0e00-4c26-bf16-2476a0da4fea} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:41604). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002482f8c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002482ea7f 5. DB::WriteBuffer::next() @ 0x000000001802d58c 6. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x0000000033459fe3 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:270: DB::TCPHandler::runImpl() @ 0x0000000033439ff9 8. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 12. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:12.434123 [ 2037 ] {9b2ff780-0e00-4c26-bf16-2476a0da4fea} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:41604). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000002482f8c7 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000002482ea7f 5. DB::WriteBuffer::next() @ 0x000000001802d58c 6. ./build_docker/./src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x0000000033458d3b 7. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x00000000334585fd 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000033439b9d 9. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 13. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 14. ? @ 0x00007fadc2097ac3 15. ? @ 0x00007fadc2129850 2024.08.31 19:53:12.434257 [ 2037 ] {9b2ff780-0e00-4c26-bf16-2476a0da4fea} TCPHandler: Client has gone away. 2024.08.31 19:53:13.995558 [ 2037 ] {8eeb1b78-982d-4c5d-a03d-1e2306958452} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.098309144 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:41764) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.046772 [ 2037 ] {8eeb1b78-982d-4c5d-a03d-1e2306958452} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.098309144 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.093105 [ 395 ] {28b7d0e6-358f-4477-ac7e-f29ac84264a5} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.091172852 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:35046) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.130843 [ 1767 ] {187a79ca-9bd4-4868-8fba-354ea016e8b3} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.249978969 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:45830) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:103: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350aea0 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 10. ? @ 0x00007fadc2097ac3 11. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.131035 [ 395 ] {28b7d0e6-358f-4477-ac7e-f29ac84264a5} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.091172852 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.131081 [ 2036 ] {5b0b7273-45bf-4837-83ba-add55a472fde} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044007089 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:39182) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d225a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.132588 [ 2035 ] {09137db9-8db2-48b9-8fb2-3fbd92ca959c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088007388 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:39336) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d225a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.134400 [ 1767 ] {187a79ca-9bd4-4868-8fba-354ea016e8b3} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.249978969 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:103: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350aea0 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 10. ? @ 0x00007fadc2097ac3 11. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.138576 [ 2035 ] {09137db9-8db2-48b9-8fb2-3fbd92ca959c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088007388 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d225a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.141022 [ 2042 ] {1cae11fb-aa4e-4cd6-8ef5-ec1e77f87fdc} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.264574603 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:46620) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.144969 [ 1736 ] {fa92db98-8df7-4323-b307-a5751326a98f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.269936977 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:54392) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.146736 [ 2036 ] {5b0b7273-45bf-4837-83ba-add55a472fde} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044007089 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d225a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.157833 [ 2042 ] {1cae11fb-aa4e-4cd6-8ef5-ec1e77f87fdc} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.264574603 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.158787 [ 2039 ] {3ad842e3-923d-443b-9768-a4d54478ae67} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.258213859 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:41606) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.163820 [ 1736 ] {fa92db98-8df7-4323-b307-a5751326a98f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.269936977 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.177530 [ 2039 ] {3ad842e3-923d-443b-9768-a4d54478ae67} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.258213859 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.546515 [ 1613 ] {ca197d0e-488a-4087-822d-500b69fa56e4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.670590932 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::ffff:127.0.0.1]:37920) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.562409 [ 2041 ] {cf84cefc-bc5b-484d-90d4-353cf5e85fe4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.012006872 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED) (version 24.3.5.48.altinityfips (altinity build)) (from [::1]:59396) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d225a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.563574 [ 2036 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:39182). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:14.563942 [ 395 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:35046). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:14.564108 [ 2039 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:41606). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:14.564308 [ 1767 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:45830). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:14.564473 [ 2037 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:41764). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:14.564713 [ 1613 ] {ca197d0e-488a-4087-822d-500b69fa56e4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.670590932 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::checkTimeLimit() @ 0x000000003350b194 6. ./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x000000003350b3f9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003350af9a 8. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000033524571 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 11. ? @ 0x00007fadc2097ac3 12. ? @ 0x00007fadc2129850 2024.08.31 19:53:14.564920 [ 2035 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:39336). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:14.565038 [ 1613 ] {ca197d0e-488a-4087-822d-500b69fa56e4} TCPHandler: Client has gone away. 2024.08.31 19:53:14.565054 [ 1736 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54392). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:14.565225 [ 2042 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:46620). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000002482ade7 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000024829bef 5. DB::ReadBuffer::next() @ 0x000000001872a050 6. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x0000000033437d41 7. ./build_docker/./src/Server/TCPHandler.cpp:2331: DB::TCPHandler::run() @ 0x0000000033473c16 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x00000000367670fe 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003676821a 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x00000000369d4030 11. ./base/poco/Foundation/include/Poco/AutoPtr.h:205: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000369cf131 12. ? @ 0x00007fadc2097ac3 13. ? @ 0x00007fadc2129850 (version 24.3.5.48.altinityfips (altinity build)) 2024.08.31 19:53:14.587435 [ 2041 ] {cf84cefc-bc5b-484d-90d4-353cf5e85fe4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.012006872 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. std::exception::capture() @ 0x0000000018191275 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:28: Poco::Exception::Exception(String const&, int) @ 0x0000000036921c45 2. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000246243cb 3. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002f3d38c2 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002f3d323e 5. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000002f3d225a 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003351eed9 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003350d4fb 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003350e0a5 9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000247328db 10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000024737934 11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl::worker(std::__list_iterator) @ 0x000000002472ff3b 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000024734c8a 13. ? @ 0x00007fadc2097ac3 14. ? @ 0x00007fadc2129850 2024.08.31 19:54:54.638267 [ 391 ] {} Application: Closed all listening sockets. Waiting for 4 outstanding connections. 2024.08.31 19:54:59.543987 [ 391 ] {} Application: Closed connections. But 4 remain. Tip: To increase wait time add to config: 60 2024.08.31 19:54:59.544065 [ 391 ] {} Application: Will shutdown forcefully.