2024.07.22 18:02:09.372947 [ 389 ] {} Context: Server was built in debug mode. It will work slowly. 2024.07.22 18:02:09.403561 [ 389 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2024.07.22 18:02:09.406691 [ 389 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.07.22 18:02:13.382558 [ 389 ] {} CloudPlacementInfo: Placement info has not been loaded 2024.07.22 18:02:13.757926 [ 389 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2024.07.22 18:02:13.758859 [ 389 ] {} KeeperLogStore: Removing all changelogs 2024.07.22 18:02:13.760241 [ 389 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.07.22 18:02:13.765357 [ 389 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.07.22 18:02:13.765474 [ 389 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.07.22 18:02:13.816973 [ 1006 ] {} RaftInstance: Election timeout, initiate leader election 2024.07.22 18:02:13.919837 [ 389 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2024.07.22 18:02:13.919964 [ 389 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2024.07.22 18:02:14.534958 [ 389 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.07.22 18:02:18.340817 [ 393 ] {94ae9a45-bbed-43e2-a198-3c33d377e62e} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.07.22 18:02:27.098259 [ 393 ] {6dbd33d1-2354-452f-9669-90af22449953} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.07.22 18:08:35.571471 [ 1576 ] {ad42e049-ce90-4c64-b275-00b24b1dad37} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:55852) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000019eabc39 8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:258: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x000000001dd373c3 9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1971: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd490fe 10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000001dcf3db0 11. /build/src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd1a27f 12. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000001b4f2661 13. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000001b4f23bb 14. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:2108: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b0beb28 15. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6510: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b08920d 16. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6622: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b087321 17. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:5540: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b09d398 18. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6473: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b088dea 19. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6622: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b087321 20. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6880: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b090b47 21. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:8115: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b083643 22. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::(anonymous namespace)::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x000000001b081874 23. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:8350: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000001b081533 24. /build/src/Analyzer/QueryTreePassManager.cpp:186: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000001b072ca6 25. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:142: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000001b4f1ccc 26. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000001b4f0906 27. /build/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000001b4f3cd7 28. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0::operator()(DB::InterpreterFactory::Arguments const&) const @ 0x000000001b4f316c 29. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001b4f311d 30. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:470: ? @ 0x000000001b4f30bd 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001b4f307d 2024.07.22 18:08:35.572356 [ 1576 ] {ad42e049-ce90-4c64-b275-00b24b1dad37} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000019eabc39 8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:258: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x000000001dd373c3 9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1971: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd490fe 10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000001dcf3db0 11. /build/src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd1a27f 12. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000001b4f2661 13. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000001b4f23bb 14. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:2108: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b0beb28 15. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6510: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b08920d 16. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6622: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b087321 17. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:5540: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b09d398 18. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6473: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b088dea 19. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6622: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b087321 20. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6880: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b090b47 21. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:8115: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b083643 22. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::(anonymous namespace)::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x000000001b081874 23. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:8350: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000001b081533 24. /build/src/Analyzer/QueryTreePassManager.cpp:186: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000001b072ca6 25. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:142: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000001b4f1ccc 26. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000001b4f0906 27. /build/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000001b4f3cd7 28. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0::operator()(DB::InterpreterFactory::Arguments const&) const @ 0x000000001b4f316c 29. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001b4f311d 30. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:470: ? @ 0x000000001b4f30bd 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001b4f307d 2024.07.22 18:08:35.781755 [ 1576 ] {8e8a4fb8-6dd0-4b3c-9efc-7c92ad7192f0} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:55852) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000019eabc39 8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:258: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x000000001dd373c3 9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1971: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd490fe 10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000001dcf3db0 11. /build/src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd1a27f 12. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000001b4f2661 13. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000001b4f23bb 14. /build/src/Interpreters/executeQuery.cpp:1195: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000001bb92eb7 15. /build/src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000001bb8cf8a 16. /build/src/Server/TCPHandler.cpp:517: DB::TCPHandler::runImpl() @ 0x000000001d45bd00 17. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 18. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 19. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 20. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 21. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 22. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 23. ? @ 0x00007fd028ac6ac3 24. ? @ 0x00007fd028b58850 2024.07.22 18:08:35.782759 [ 1576 ] {8e8a4fb8-6dd0-4b3c-9efc-7c92ad7192f0} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000019eabc39 8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:258: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x000000001dd373c3 9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1971: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd490fe 10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000001dcf3db0 11. /build/src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd1a27f 12. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000001b4f2661 13. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000001b4f23bb 14. /build/src/Interpreters/executeQuery.cpp:1195: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000001bb92eb7 15. /build/src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000001bb8cf8a 16. /build/src/Server/TCPHandler.cpp:517: DB::TCPHandler::runImpl() @ 0x000000001d45bd00 17. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 18. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 19. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 20. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 21. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 22. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 23. ? @ 0x00007fd028ac6ac3 24. ? @ 0x00007fd028b58850 2024.07.22 18:08:37.137165 [ 1576 ] {355f2070-0be5-4ee7-aba0-984237b82f18} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 15.36 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:55852) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:08:37.213590 [ 1576 ] {355f2070-0be5-4ee7-aba0-984237b82f18} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 15.36 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:08:37.466290 [ 1576 ] {f37b0846-c964-4659-80a7-b50d2ba84190} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:55852) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000019eabc39 8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:258: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x000000001dd373c3 9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1971: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd490fe 10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000001dcf3db0 11. /build/src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd1a27f 12. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000001b4f2661 13. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000001b4f23bb 14. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:2108: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b0beb28 15. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6510: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b08920d 16. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6622: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b087321 17. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:5540: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b09d398 18. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6473: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b088dea 19. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:8140: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b083977 20. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::(anonymous namespace)::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x000000001b081874 21. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:8350: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000001b081533 22. /build/src/Analyzer/QueryTreePassManager.cpp:186: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000001b072ca6 23. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:142: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000001b4f1ccc 24. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000001b4f0906 25. /build/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000001b4f3cd7 26. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0::operator()(DB::InterpreterFactory::Arguments const&) const @ 0x000000001b4f316c 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001b4f311d 28. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:470: ? @ 0x000000001b4f30bd 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001b4f307d 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b4f3045 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x000000001b40e19e 2024.07.22 18:08:37.467329 [ 1576 ] {f37b0846-c964-4659-80a7-b50d2ba84190} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000019eabc39 8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:258: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x000000001dd373c3 9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1971: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd490fe 10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000001dcf3db0 11. /build/src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000001dd1a27f 12. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000001b4f2661 13. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:221: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000001b4f23bb 14. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:2108: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b0beb28 15. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6510: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b08920d 16. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6622: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b087321 17. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:5540: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b09d398 18. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:6473: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001b088dea 19. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:8140: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001b083977 20. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:1148: DB::(anonymous namespace)::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x000000001b081874 21. /build/src/Analyzer/Passes/QueryAnalysisPass.cpp:8350: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000001b081533 22. /build/src/Analyzer/QueryTreePassManager.cpp:186: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000001b072ca6 23. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:142: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000001b4f1ccc 24. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000001b4f0906 25. /build/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15000]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000001b4f3cd7 26. /build/src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0::operator()(DB::InterpreterFactory::Arguments const&) const @ 0x000000001b4f316c 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001b4f311d 28. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:470: ? @ 0x000000001b4f30bd 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001b4f307d 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b4f3045 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x000000001b40e19e 2024.07.22 18:08:38.307950 [ 1576 ] {c03e84f7-ea3b-45a6-a4e0-f81fb1c1fcb2} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.25 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:55852) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:08:38.321615 [ 1576 ] {c03e84f7-ea3b-45a6-a4e0-f81fb1c1fcb2} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.25 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x0000000019eabdc8 6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eab8ec 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:17:59.331196 [ 2828 ] {f0538282-1c05-477a-859e-b69550afe0c0} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.80 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:35638) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000019eabfe8 6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eabac7 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:17:59.372148 [ 2828 ] {f0538282-1c05-477a-859e-b69550afe0c0} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.80 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000019eabfe8 6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eabac7 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:17:59.886487 [ 2828 ] {75c5e799-85ec-4f94-a24b-eae0ca6f9751} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 582.02 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:35638) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000019eabfe8 6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eabac7 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:17:59.927531 [ 2828 ] {75c5e799-85ec-4f94-a24b-eae0ca6f9751} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 582.02 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000019eabfe8 6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eabac7 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:18:00.464605 [ 2828 ] {bfa40c10-40ac-4ed2-950d-f51ae91c2446} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 581.66 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:35638) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000019eabfe8 6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eabac7 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:18:00.506008 [ 2828 ] {bfa40c10-40ac-4ed2-950d-f51ae91c2446} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 581.66 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000019eabfe8 6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000019eabac7 7. /build/src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000019e7dd24 8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x000000001d51fad1 9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:95: DB::ExecutionThreadContext::executeTask() @ 0x000000001d51f580 10. /build/src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000001d501b01 11. /build/src/Processors/Executors/PipelineExecutor.cpp:239: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001d501e97 12. /build/src/Processors/Executors/PipelineExecutor.cpp:373: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x000000001d502e38 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d502d95 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d502d75 15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d502d55 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d502d20 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 19. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x00000000133ca61d 20. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133d3bc4 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3b95 22. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x00000000133d3b79 23. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x00000000133d3a1d 24. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x00000000133d38e5 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133d3855 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x00000000133d3835 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x00000000133d3815 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000133d37e0 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 31. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 2024.07.22 18:18:09.551229 [ 2830 ] {2ffd8db0-2cef-4c2d-bd8c-e94402c7d792} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.132716703 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:42282) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:09.736834 [ 1989 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:52086). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x00000000134d65ec 6. /build/src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x00000000134d5760 7. /build/src/IO/ReadBuffer.h:70: DB::ReadBuffer::next() @ 0x000000000af53aea 8. /build/src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000001d45a917 9. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 10. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 11. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 12. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 13. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 14. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 15. ? @ 0x00007fd028ac6ac3 16. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:09.736909 [ 2830 ] {2ffd8db0-2cef-4c2d-bd8c-e94402c7d792} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.132716703 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:14.205031 [ 1989 ] {58bed619-6d5c-4b14-bc44-7545c5ea7d8a} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088390815 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:42292) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:14.331724 [ 2830 ] {9a16e097-1cd3-4a81-bd9d-dad2483fea8e} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41044). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:41044) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1055: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d466e39 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:14.331784 [ 1989 ] {58bed619-6d5c-4b14-bc44-7545c5ea7d8a} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088390815 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:14.420806 [ 1987 ] {9ae24bfc-74e9-4432-9ba5-f061823f7a5e} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:52096). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52096) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:14.527408 [ 2830 ] {9a16e097-1cd3-4a81-bd9d-dad2483fea8e} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41044). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:14.527460 [ 1987 ] {9ae24bfc-74e9-4432-9ba5-f061823f7a5e} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:52096). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:14.528091 [ 1987 ] {9ae24bfc-74e9-4432-9ba5-f061823f7a5e} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:52096). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:14.528144 [ 2830 ] {9a16e097-1cd3-4a81-bd9d-dad2483fea8e} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41044). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1055: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d466e39 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:14.528628 [ 1987 ] {9ae24bfc-74e9-4432-9ba5-f061823f7a5e} TCPHandler: Client has gone away. 2024.07.22 18:18:14.528893 [ 2830 ] {9a16e097-1cd3-4a81-bd9d-dad2483fea8e} TCPHandler: Client has gone away. 2024.07.22 18:18:18.873724 [ 1989 ] {e5b4aa99-0474-41a8-ac86-019a8d9a7100} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.103902928 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:55952) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:18.927708 [ 2830 ] {ecc621af-1b2a-48ff-a0eb-7a5b27f8be78} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55336). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55336) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:19.131590 [ 3257 ] {a40a349a-d726-4b14-8ba2-07dd675cfe9c} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:45042). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45042) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:19.131614 [ 1989 ] {e5b4aa99-0474-41a8-ac86-019a8d9a7100} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.103902928 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:19.131805 [ 2830 ] {ecc621af-1b2a-48ff-a0eb-7a5b27f8be78} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55336). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:19.132335 [ 2830 ] {ecc621af-1b2a-48ff-a0eb-7a5b27f8be78} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55336). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:19.133164 [ 2830 ] {ecc621af-1b2a-48ff-a0eb-7a5b27f8be78} TCPHandler: Client has gone away. 2024.07.22 18:18:19.136570 [ 1987 ] {def5bd83-5cac-4006-8d48-f3ca8c03d1e4} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:54170). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54170) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:19.182556 [ 1987 ] {def5bd83-5cac-4006-8d48-f3ca8c03d1e4} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:54170). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:19.183046 [ 1987 ] {def5bd83-5cac-4006-8d48-f3ca8c03d1e4} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:54170). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:19.183787 [ 1987 ] {def5bd83-5cac-4006-8d48-f3ca8c03d1e4} TCPHandler: Client has gone away. 2024.07.22 18:18:19.188720 [ 3257 ] {a40a349a-d726-4b14-8ba2-07dd675cfe9c} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:45042). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:19.189201 [ 3257 ] {a40a349a-d726-4b14-8ba2-07dd675cfe9c} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:45042). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:19.189737 [ 3257 ] {a40a349a-d726-4b14-8ba2-07dd675cfe9c} TCPHandler: Client has gone away. 2024.07.22 18:18:23.471512 [ 1987 ] {2c296e1d-06ba-4902-acf6-9d266a7250fc} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.076321719 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:55960) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.514383 [ 2830 ] {59b6f11c-6ab2-46a9-a6e4-0651e597caab} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55352). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55352) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.515062 [ 2829 ] {b857a213-db90-43bd-9cd6-a790a5341d31} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:54182). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54182) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.569082 [ 1987 ] {2c296e1d-06ba-4902-acf6-9d266a7250fc} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.076321719 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.569347 [ 3257 ] {3e1ed6c0-7d16-428a-b293-5e51b11ba460} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:45046). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45046) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.575878 [ 2829 ] {b857a213-db90-43bd-9cd6-a790a5341d31} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:54182). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:23.576328 [ 2829 ] {b857a213-db90-43bd-9cd6-a790a5341d31} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:54182). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.576710 [ 3267 ] {51adc6e1-1922-4625-9ecb-ea0f7c61c5b0} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37428). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37428) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.576869 [ 2829 ] {b857a213-db90-43bd-9cd6-a790a5341d31} TCPHandler: Client has gone away. 2024.07.22 18:18:23.578600 [ 2830 ] {59b6f11c-6ab2-46a9-a6e4-0651e597caab} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55352). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:23.579225 [ 2830 ] {59b6f11c-6ab2-46a9-a6e4-0651e597caab} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:55352). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.579849 [ 2830 ] {59b6f11c-6ab2-46a9-a6e4-0651e597caab} TCPHandler: Client has gone away. 2024.07.22 18:18:23.613787 [ 3257 ] {3e1ed6c0-7d16-428a-b293-5e51b11ba460} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:45046). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:23.614311 [ 3257 ] {3e1ed6c0-7d16-428a-b293-5e51b11ba460} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:45046). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.614912 [ 3257 ] {3e1ed6c0-7d16-428a-b293-5e51b11ba460} TCPHandler: Client has gone away. 2024.07.22 18:18:23.625608 [ 3267 ] {51adc6e1-1922-4625-9ecb-ea0f7c61c5b0} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37428). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:23.626033 [ 3267 ] {51adc6e1-1922-4625-9ecb-ea0f7c61c5b0} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37428). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:23.626606 [ 3267 ] {51adc6e1-1922-4625-9ecb-ea0f7c61c5b0} TCPHandler: Client has gone away. 2024.07.22 18:18:27.912775 [ 2828 ] {3344a0b6-c5e1-496d-884f-f46c72cc530c} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.083968053 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60204) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:27.953795 [ 2829 ] {89dc1bef-5b7a-4b82-a758-4ccd8b091194} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:36540). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36540) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:27.963867 [ 3257 ] {f5de048e-f060-4bd5-bd10-0f7d19fd291d} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:57556). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57556) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:27.969226 [ 3277 ] {8933ae67-a1d1-42e1-b5ea-8d8aacdfa475} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:55574). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55574) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:27.972606 [ 3267 ] {3e4a94d7-ad70-49b1-80fe-f5ef478dae12} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52384). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52384) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:28.014696 [ 2830 ] {71ebd0f8-e18f-4d47-910d-df71711e298c} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:38082). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38082) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:28.021542 [ 2828 ] {3344a0b6-c5e1-496d-884f-f46c72cc530c} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.083968053 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:28.023275 [ 2829 ] {89dc1bef-5b7a-4b82-a758-4ccd8b091194} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:36540). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:28.023719 [ 2829 ] {89dc1bef-5b7a-4b82-a758-4ccd8b091194} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:36540). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:28.024248 [ 2829 ] {89dc1bef-5b7a-4b82-a758-4ccd8b091194} TCPHandler: Client has gone away. 2024.07.22 18:18:28.025620 [ 3257 ] {f5de048e-f060-4bd5-bd10-0f7d19fd291d} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:57556). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:28.026018 [ 3257 ] {f5de048e-f060-4bd5-bd10-0f7d19fd291d} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:57556). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:28.026797 [ 3257 ] {f5de048e-f060-4bd5-bd10-0f7d19fd291d} TCPHandler: Client has gone away. 2024.07.22 18:18:28.028799 [ 3277 ] {8933ae67-a1d1-42e1-b5ea-8d8aacdfa475} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:55574). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:28.029545 [ 3277 ] {8933ae67-a1d1-42e1-b5ea-8d8aacdfa475} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:55574). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:28.030242 [ 3277 ] {8933ae67-a1d1-42e1-b5ea-8d8aacdfa475} TCPHandler: Client has gone away. 2024.07.22 18:18:28.043244 [ 3267 ] {3e4a94d7-ad70-49b1-80fe-f5ef478dae12} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52384). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:28.043677 [ 3267 ] {3e4a94d7-ad70-49b1-80fe-f5ef478dae12} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52384). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:28.044247 [ 3267 ] {3e4a94d7-ad70-49b1-80fe-f5ef478dae12} TCPHandler: Client has gone away. 2024.07.22 18:18:28.079750 [ 2830 ] {71ebd0f8-e18f-4d47-910d-df71711e298c} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:38082). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:28.080186 [ 2830 ] {71ebd0f8-e18f-4d47-910d-df71711e298c} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:38082). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:28.080657 [ 2830 ] {71ebd0f8-e18f-4d47-910d-df71711e298c} TCPHandler: Client has gone away. 2024.07.22 18:18:32.357822 [ 2828 ] {e5945e52-81d6-4c3d-97f6-4285ae69a3e5} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.092120675 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60228) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.392044 [ 3288 ] {c103b172-d165-4a43-b200-6192fa130cd4} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:57566). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57566) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.409337 [ 2829 ] {4e0d78b7-0003-4fd9-97c3-c845c802015e} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:38084). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38084) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.419461 [ 3267 ] {742624be-4ee2-4970-b4c5-de3a4a9a52a7} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:55584). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55584) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.441712 [ 2828 ] {e5945e52-81d6-4c3d-97f6-4285ae69a3e5} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.092120675 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.445160 [ 3257 ] {22fb2c8a-9fee-480a-8905-4fe2d9fa1239} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52394). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52394) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.449575 [ 3277 ] {f91a7990-95bb-48d3-88f2-62beae7d500b} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:36556). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36556) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.451061 [ 3288 ] {c103b172-d165-4a43-b200-6192fa130cd4} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:57566). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:32.451677 [ 3288 ] {c103b172-d165-4a43-b200-6192fa130cd4} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:57566). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.452584 [ 3288 ] {c103b172-d165-4a43-b200-6192fa130cd4} TCPHandler: Client has gone away. 2024.07.22 18:18:32.459569 [ 2830 ] {2c3be7a0-eff6-4908-a31f-33a5355dd7a3} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:33886). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33886) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.480954 [ 3267 ] {742624be-4ee2-4970-b4c5-de3a4a9a52a7} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:55584). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:32.481427 [ 3267 ] {742624be-4ee2-4970-b4c5-de3a4a9a52a7} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:55584). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.481970 [ 3267 ] {742624be-4ee2-4970-b4c5-de3a4a9a52a7} TCPHandler: Client has gone away. 2024.07.22 18:18:32.490580 [ 2829 ] {4e0d78b7-0003-4fd9-97c3-c845c802015e} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:38084). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:32.491062 [ 2829 ] {4e0d78b7-0003-4fd9-97c3-c845c802015e} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:38084). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.491790 [ 2829 ] {4e0d78b7-0003-4fd9-97c3-c845c802015e} TCPHandler: Client has gone away. 2024.07.22 18:18:32.497153 [ 3257 ] {22fb2c8a-9fee-480a-8905-4fe2d9fa1239} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52394). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:32.497626 [ 3257 ] {22fb2c8a-9fee-480a-8905-4fe2d9fa1239} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52394). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.498183 [ 3257 ] {22fb2c8a-9fee-480a-8905-4fe2d9fa1239} TCPHandler: Client has gone away. 2024.07.22 18:18:32.517763 [ 2830 ] {2c3be7a0-eff6-4908-a31f-33a5355dd7a3} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:33886). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:32.518492 [ 2830 ] {2c3be7a0-eff6-4908-a31f-33a5355dd7a3} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:33886). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.519245 [ 2830 ] {2c3be7a0-eff6-4908-a31f-33a5355dd7a3} TCPHandler: Client has gone away. 2024.07.22 18:18:32.532202 [ 3277 ] {f91a7990-95bb-48d3-88f2-62beae7d500b} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:36556). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:32.532714 [ 3277 ] {f91a7990-95bb-48d3-88f2-62beae7d500b} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:36556). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:32.533282 [ 3277 ] {f91a7990-95bb-48d3-88f2-62beae7d500b} TCPHandler: Client has gone away. 2024.07.22 18:18:36.863034 [ 2829 ] {8e50e175-7e89-44df-82d4-53fbdf327fe1} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.105672129 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60562) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:36.928454 [ 2830 ] {e1c180b1-7323-454a-919d-09963d296a63} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:60566). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60566) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:36.939975 [ 3257 ] {5f14f18f-fb4b-4ca2-8589-fcdc61f90068} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:51036). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51036) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:36.940285 [ 3267 ] {738eedeb-4884-418e-8322-2f6d734ebd3c} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:37596). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37596) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:36.945981 [ 3277 ] {bdf4e51c-1aca-416c-8a35-1ac46dde5ed3} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:56242). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56242) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:36.956429 [ 3288 ] {82461a40-058d-4c58-ba10-d63a9480e167} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:52710). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52710) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:36.956895 [ 1989 ] {304c4102-2e6e-48a5-bd3c-f2e5e107d52e} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39606). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39606) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:36.968203 [ 3301 ] {b04e3f28-fb9e-4559-a91a-d1ba44f0e7d1} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:46244). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:46244) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:36.997356 [ 2830 ] {e1c180b1-7323-454a-919d-09963d296a63} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:60566). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:36.997959 [ 2830 ] {e1c180b1-7323-454a-919d-09963d296a63} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:60566). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:36.998697 [ 2830 ] {e1c180b1-7323-454a-919d-09963d296a63} TCPHandler: Client has gone away. 2024.07.22 18:18:37.011811 [ 1989 ] {304c4102-2e6e-48a5-bd3c-f2e5e107d52e} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39606). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:37.012028 [ 3267 ] {738eedeb-4884-418e-8322-2f6d734ebd3c} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:37596). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:37.012385 [ 1989 ] {304c4102-2e6e-48a5-bd3c-f2e5e107d52e} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39606). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:37.012633 [ 3267 ] {738eedeb-4884-418e-8322-2f6d734ebd3c} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:37596). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:37.013401 [ 1989 ] {304c4102-2e6e-48a5-bd3c-f2e5e107d52e} TCPHandler: Client has gone away. 2024.07.22 18:18:37.013427 [ 3267 ] {738eedeb-4884-418e-8322-2f6d734ebd3c} TCPHandler: Client has gone away. 2024.07.22 18:18:37.015814 [ 2829 ] {8e50e175-7e89-44df-82d4-53fbdf327fe1} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.105672129 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:37.016572 [ 3257 ] {5f14f18f-fb4b-4ca2-8589-fcdc61f90068} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:51036). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:37.016990 [ 3257 ] {5f14f18f-fb4b-4ca2-8589-fcdc61f90068} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:51036). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:37.017654 [ 3257 ] {5f14f18f-fb4b-4ca2-8589-fcdc61f90068} TCPHandler: Client has gone away. 2024.07.22 18:18:37.021575 [ 3277 ] {bdf4e51c-1aca-416c-8a35-1ac46dde5ed3} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:56242). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:37.022023 [ 3277 ] {bdf4e51c-1aca-416c-8a35-1ac46dde5ed3} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:56242). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:37.022556 [ 3277 ] {bdf4e51c-1aca-416c-8a35-1ac46dde5ed3} TCPHandler: Client has gone away. 2024.07.22 18:18:37.025239 [ 3288 ] {82461a40-058d-4c58-ba10-d63a9480e167} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:52710). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:37.025761 [ 3288 ] {82461a40-058d-4c58-ba10-d63a9480e167} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:52710). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:37.026496 [ 3288 ] {82461a40-058d-4c58-ba10-d63a9480e167} TCPHandler: Client has gone away. 2024.07.22 18:18:37.036783 [ 3301 ] {b04e3f28-fb9e-4559-a91a-d1ba44f0e7d1} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:46244). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:37.037172 [ 3301 ] {b04e3f28-fb9e-4559-a91a-d1ba44f0e7d1} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:46244). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:37.037666 [ 3301 ] {b04e3f28-fb9e-4559-a91a-d1ba44f0e7d1} TCPHandler: Client has gone away. 2024.07.22 18:18:41.608934 [ 2829 ] {60145947-664b-4474-bd94-8369b639cc58} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.359462288 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:60586) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.628654 [ 2830 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:46248). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x00000000134d65ec 6. /build/src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x00000000134d5760 7. /build/src/IO/ReadBuffer.h:70: DB::ReadBuffer::next() @ 0x000000000af53aea 8. /build/src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000001d45a917 9. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 10. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 11. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 12. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 13. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 14. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 15. ? @ 0x00007fd028ac6ac3 16. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:41.629990 [ 1989 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:60576). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x00000000134d65ec 6. /build/src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x00000000134d5760 7. /build/src/IO/ReadBuffer.h:70: DB::ReadBuffer::next() @ 0x000000000af53aea 8. /build/src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000001d45a917 9. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 10. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 11. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 12. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 13. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 14. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 15. ? @ 0x00007fd028ac6ac3 16. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:41.638052 [ 3267 ] {159c3760-dfd9-4058-ae54-7e00afd41562} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:52712). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52712) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.744672 [ 3288 ] {8484802e-4a6e-412b-8893-b23246cc57c8} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:37604). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37604) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.745046 [ 3267 ] {159c3760-dfd9-4058-ae54-7e00afd41562} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:52712). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:41.745536 [ 3267 ] {159c3760-dfd9-4058-ae54-7e00afd41562} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:52712). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.746281 [ 3267 ] {159c3760-dfd9-4058-ae54-7e00afd41562} TCPHandler: Client has gone away. 2024.07.22 18:18:41.747214 [ 2829 ] {60145947-664b-4474-bd94-8369b639cc58} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.359462288 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.744677 [ 3257 ] {06d249c7-7082-4f59-9bff-b46ed0544def} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:51044). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51044) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.750092 [ 3257 ] {06d249c7-7082-4f59-9bff-b46ed0544def} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:51044). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:41.750530 [ 3257 ] {06d249c7-7082-4f59-9bff-b46ed0544def} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:51044). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.751164 [ 3257 ] {06d249c7-7082-4f59-9bff-b46ed0544def} TCPHandler: Client has gone away. 2024.07.22 18:18:41.916445 [ 3277 ] {3d53f02e-d8a6-4474-bf9b-3c71e10ef2fb} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:35958). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35958) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.916590 [ 3288 ] {8484802e-4a6e-412b-8893-b23246cc57c8} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:37604). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:41.917226 [ 3288 ] {8484802e-4a6e-412b-8893-b23246cc57c8} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:37604). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.917845 [ 3288 ] {8484802e-4a6e-412b-8893-b23246cc57c8} TCPHandler: Client has gone away. 2024.07.22 18:18:41.955507 [ 3277 ] {3d53f02e-d8a6-4474-bf9b-3c71e10ef2fb} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:35958). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:41.955899 [ 3277 ] {3d53f02e-d8a6-4474-bf9b-3c71e10ef2fb} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:35958). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:41.956387 [ 3277 ] {3d53f02e-d8a6-4474-bf9b-3c71e10ef2fb} TCPHandler: Client has gone away. 2024.07.22 18:18:46.610588 [ 2829 ] {e701e009-5b2c-4143-bbd6-a8c2d23923a7} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.485297142 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.5.47.altinitystable (altinity build)) (from [::1]:47692) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.633423 [ 1989 ] {6f749c82-12e5-4fd0-8c58-e08823ea1915} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:47152). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47152) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.635394 [ 3312 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:47282). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x00000000134d65ec 6. /build/src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x00000000134d5760 7. /build/src/IO/ReadBuffer.h:70: DB::ReadBuffer::next() @ 0x000000000af53aea 8. /build/src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000001d45a917 9. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 10. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 11. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 12. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 13. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 14. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 15. ? @ 0x00007fd028ac6ac3 16. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:46.650167 [ 3301 ] {ab9e65c8-a6cd-46fc-b5c8-28d585b029d6} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:38818). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38818) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.694298 [ 2830 ] {bf6e14ba-a511-4eca-835e-120d58074fbd} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:47770). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47770) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.704290 [ 3277 ] {e1c50c1a-7679-43ea-983a-bfd9db89215a} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:43958). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:43958) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.709067 [ 3277 ] {e1c50c1a-7679-43ea-983a-bfd9db89215a} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:43958). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:46.709655 [ 3277 ] {e1c50c1a-7679-43ea-983a-bfd9db89215a} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:43958). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.710421 [ 3277 ] {e1c50c1a-7679-43ea-983a-bfd9db89215a} TCPHandler: Client has gone away. 2024.07.22 18:18:46.710757 [ 2830 ] {bf6e14ba-a511-4eca-835e-120d58074fbd} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:47770). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:46.711371 [ 2830 ] {bf6e14ba-a511-4eca-835e-120d58074fbd} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:47770). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.712232 [ 2830 ] {bf6e14ba-a511-4eca-835e-120d58074fbd} TCPHandler: Client has gone away. 2024.07.22 18:18:46.712471 [ 3313 ] {e65cbc1c-8220-4728-8321-5779f0126e33} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:45746). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45746) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.714448 [ 3313 ] {e65cbc1c-8220-4728-8321-5779f0126e33} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:45746). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:46.714824 [ 3313 ] {e65cbc1c-8220-4728-8321-5779f0126e33} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:45746). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.715359 [ 3313 ] {e65cbc1c-8220-4728-8321-5779f0126e33} TCPHandler: Client has gone away. 2024.07.22 18:18:46.718446 [ 3288 ] {52fe4833-baaa-42d5-a731-f7764f5dc0a6} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:48110). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48110) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.724281 [ 3267 ] {be586658-0ee3-40bd-bc39-0125f1ec136c} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:41878). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:41878) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.728900 [ 3267 ] {be586658-0ee3-40bd-bc39-0125f1ec136c} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:41878). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:46.729642 [ 3267 ] {be586658-0ee3-40bd-bc39-0125f1ec136c} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:41878). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.730403 [ 3267 ] {be586658-0ee3-40bd-bc39-0125f1ec136c} TCPHandler: Client has gone away. 2024.07.22 18:18:46.739292 [ 3257 ] {184b776e-4217-4503-8543-c5b0865ccd89} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:54930). (NETWORK_ERROR) (version 24.3.5.47.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54930) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.741148 [ 3257 ] {184b776e-4217-4503-8543-c5b0865ccd89} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:54930). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:46.741581 [ 3257 ] {184b776e-4217-4503-8543-c5b0865ccd89} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:54930). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2273: DB::TCPHandler::sendEndOfStream() @ 0x000000001d467d83 9. /build/src/Server/TCPHandler.cpp:596: DB::TCPHandler::runImpl() @ 0x000000001d45c1b5 10. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 16. ? @ 0x00007fd028ac6ac3 17. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.742173 [ 3257 ] {184b776e-4217-4503-8543-c5b0865ccd89} TCPHandler: Client has gone away. 2024.07.22 18:18:46.743856 [ 3301 ] {ab9e65c8-a6cd-46fc-b5c8-28d585b029d6} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:38818). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:46.744281 [ 3301 ] {ab9e65c8-a6cd-46fc-b5c8-28d585b029d6} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:38818). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.744829 [ 3301 ] {ab9e65c8-a6cd-46fc-b5c8-28d585b029d6} TCPHandler: Client has gone away. 2024.07.22 18:18:46.756807 [ 1989 ] {6f749c82-12e5-4fd0-8c58-e08823ea1915} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:47152). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:46.757369 [ 1989 ] {6f749c82-12e5-4fd0-8c58-e08823ea1915} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:47152). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2291: DB::TCPHandler::sendProgress() @ 0x000000001d46723d 9. /build/src/Server/TCPHandler.cpp:1088: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d4670bb 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.758027 [ 1989 ] {6f749c82-12e5-4fd0-8c58-e08823ea1915} TCPHandler: Client has gone away. 2024.07.22 18:18:46.781693 [ 3288 ] {52fe4833-baaa-42d5-a731-f7764f5dc0a6} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:48110). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d8104 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2243: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001d472cb5 9. /build/src/Server/TCPHandler.cpp:2320: DB::TCPHandler::sendLogs() @ 0x000000001d467c9f 10. /build/src/Server/TCPHandler.cpp:693: DB::TCPHandler::runImpl() @ 0x000000001d45d162 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 (version 24.3.5.47.altinitystable (altinity build)) 2024.07.22 18:18:46.782376 [ 3288 ] {52fe4833-baaa-42d5-a731-f7764f5dc0a6} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:48110). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x00000000134d8c08 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x00000000134d7e64 7. /build/src/IO/WriteBuffer.h:53: DB::WriteBuffer::next() @ 0x000000000a920a6d 8. /build/src/Server/TCPHandler.cpp:2207: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001d46920f 9. /build/src/Server/TCPHandler.cpp:1084: DB::TCPHandler::processOrdinaryQuery() @ 0x000000001d46704a 10. /build/src/Server/TCPHandler.cpp:539: DB::TCPHandler::runImpl() @ 0x000000001d45befb 11. /build/src/Server/TCPHandler.cpp:2329: DB::TCPHandler::run() @ 0x000000001d472d25 12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x000000002287ed39 13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000002287f588 14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x0000000022a56874 15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x0000000022a5361a 16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000022a5235e 17. ? @ 0x00007fd028ac6ac3 18. ? @ 0x00007fd028b58850 2024.07.22 18:18:46.782958 [ 3288 ] {52fe4833-baaa-42d5-a731-f7764f5dc0a6} TCPHandler: Client has gone away. 2024.07.22 18:18:46.801682 [ 2829 ] {e701e009-5b2c-4143-bbd6-a8c2d23923a7} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.485297142 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:141: std::exception::capture() @ 0x000000000a8ef122 1. /build/contrib/llvm-project/libcxx/include/exception:116: std::exception::exception[abi:v15000]() @ 0x000000000a8ef0ed 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x00000000229d6ac0 3. /build/src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000132ad8ce 4. /build/src/Common/Exception.h:90: DB::Exception::Exception(String&&, int, bool) @ 0x000000000a8e5cca 5. /build/src/Common/Exception.h:109: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e8046c 6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:111: bool DB::handleOverflowMode(DB::OverflowMode, int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000019e7f879 7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:127: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000019e7f788 8. /build/src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001b68651e 9. /build/src/Processors/Executors/PipelineExecutor.cpp:186: DB::PipelineExecutor::checkTimeLimit() @ 0x000000001d500a20 10. /build/src/Processors/Executors/PipelineExecutor.cpp:196: DB::PipelineExecutor::finalizeExecution() @ 0x000000001d5010df 11. /build/src/Processors/Executors/PipelineExecutor.cpp:131: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001d50099e 12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr, unsigned long, bool) @ 0x000000001d5259d0 13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:110: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x000000001d52583f 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d5257b5 15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001d525799 16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x000000001d5256fd 17. /build/src/Common/ThreadPool.h:223: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x000000001d525607 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x000000001d525575 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: ? @ 0x000000001d525555 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: ? @ 0x000000001d525535 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001d525500 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000011b37036 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: ? @ 0x0000000011b36c75 24. /build/src/Common/ThreadPool.cpp:455: ThreadPoolImpl::worker(std::__list_iterator) @ 0x00000000133c73dd 25. /build/src/Common/ThreadPool.cpp:217: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x00000000133ce384 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: ? @ 0x00000000133ce335 27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x00000000133ce2fd 28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x00000000133ce102 29. ? @ 0x00007fd028ac6ac3 30. ? @ 0x00007fd028b58850 2024.07.22 18:31:59.926382 [ 389 ] {} Application: Closed all listening sockets. Waiting for 4 outstanding connections. 2024.07.22 18:32:18.808458 [ 1009 ] {} RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled