2024.11.06 01:09:39.804312 [ 373 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2024.11.06 01:09:39.804820 [ 373 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.11.06 01:09:39.804858 [ 373 ] {} Application: Integrity check of the executable skipped because the reference checksum could not be read. 2024.11.06 01:09:39.923704 [ 373 ] {} CloudPlacementInfo: Placement info has not been loaded 2024.11.06 01:09:40.011924 [ 373 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2024.11.06 01:09:40.012396 [ 373 ] {} KeeperLogStore: Removing all changelogs 2024.11.06 01:09:40.013008 [ 373 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.11.06 01:09:40.014847 [ 373 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.11.06 01:09:40.014867 [ 373 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.11.06 01:09:40.065920 [ 952 ] {} RaftInstance: Election timeout, initiate leader election 2024.11.06 01:09:40.083568 [ 373 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2024.11.06 01:09:40.083585 [ 373 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2024.11.06 01:09:40.131111 [ 373 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.11.06 01:09:41.435122 [ 376 ] {44faee2b-2dc9-4c28-8620-ae1843d9fff7} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.11.06 01:09:52.052054 [ 376 ] {310963cd-8909-403f-92b3-8d62ff432590} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2024.11.06 01:15:24.922939 [ 376 ] {bcf66525-ca6f-475e-a2fd-ed9434d5596e} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45456) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./src/QueryPipeline/SizeLimits.cpp:65: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010de4784 4. ./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dac9f4 5. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dc5a24 6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000000f7ce184 7. ./contrib/llvm-project/libcxx/include/vector:666: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000000f7cdc14 8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2111: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5c200c 9. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a4508 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a3708 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5b80d4 12. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a423c 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a3708 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5ab838 15. ./contrib/llvm-project/libcxx/include/vector:1314: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f59d50c 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1201: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000000f59b6ac 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:184: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000000f599f24 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) (.llvm.5699878785432179294) @ 0x000000000f7cce24 19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000000f7cae3c 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) (.llvm.5699878785432179294) @ 0x000000000f7ce908 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000f76fc70 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000000fb3c3a4 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000000fb38df0 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000109ac35c 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 30. ? @ 0x000000000007d5c8 31. ? @ 0x00000000000e5edc 2024.11.06 01:15:24.923153 [ 376 ] {bcf66525-ca6f-475e-a2fd-ed9434d5596e} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./src/QueryPipeline/SizeLimits.cpp:65: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010de4784 4. ./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dac9f4 5. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dc5a24 6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000000f7ce184 7. ./contrib/llvm-project/libcxx/include/vector:666: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000000f7cdc14 8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2111: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5c200c 9. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a4508 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a3708 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5b80d4 12. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a423c 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a3708 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5ab838 15. ./contrib/llvm-project/libcxx/include/vector:1314: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f59d50c 16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1201: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000000f59b6ac 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:184: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000000f599f24 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) (.llvm.5699878785432179294) @ 0x000000000f7cce24 19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000000f7cae3c 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) (.llvm.5699878785432179294) @ 0x000000000f7ce908 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000f76fc70 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000000fb3c3a4 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000000fb38df0 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000109ac35c 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 30. ? @ 0x000000000007d5c8 31. ? @ 0x00000000000e5edc 2024.11.06 01:15:24.936373 [ 376 ] {217c7f98-f074-43f7-90f0-b2274178039c} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45456) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./src/QueryPipeline/SizeLimits.cpp:65: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010de4784 4. ./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dac9f4 5. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dc5a24 6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000000f7ce184 7. ./contrib/llvm-project/libcxx/include/vector:666: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000000f7cdc14 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1195: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000000fb3c9c8 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000000fb38df0 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000109ac35c 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 16. ? @ 0x000000000007d5c8 17. ? @ 0x00000000000e5edc 2024.11.06 01:15:24.936586 [ 376 ] {217c7f98-f074-43f7-90f0-b2274178039c} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./src/QueryPipeline/SizeLimits.cpp:65: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010de4784 4. ./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dac9f4 5. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dc5a24 6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000000f7ce184 7. ./contrib/llvm-project/libcxx/include/vector:666: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000000f7cdc14 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1195: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000000fb3c9c8 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000000fb38df0 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000109ac35c 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 16. ? @ 0x000000000007d5c8 17. ? @ 0x00000000000e5edc 2024.11.06 01:15:25.000339 [ 376 ] {e8771540-4261-4d22-8eb2-57ad5ea21dda} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.84 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45456) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:15:25.004323 [ 376 ] {e8771540-4261-4d22-8eb2-57ad5ea21dda} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.84 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:15:25.024879 [ 376 ] {356852a9-4f05-4b02-acb3-fa27df00eede} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45456) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./src/QueryPipeline/SizeLimits.cpp:65: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010de4784 4. ./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dac9f4 5. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dc5a24 6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000000f7ce184 7. ./contrib/llvm-project/libcxx/include/vector:666: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000000f7cdc14 8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2111: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5c200c 9. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a4508 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a3708 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5b80d4 12. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a423c 13. ./contrib/llvm-project/libcxx/include/vector:434: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f59d67c 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1201: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000000f59b6ac 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:184: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000000f599f24 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) (.llvm.5699878785432179294) @ 0x000000000f7cce24 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000000f7cae3c 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) (.llvm.5699878785432179294) @ 0x000000000f7ce908 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000f76fc70 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000000fb3c3a4 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000000fb38df0 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000109ac35c 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 28. ? @ 0x000000000007d5c8 29. ? @ 0x00000000000e5edc 2024.11.06 01:15:25.025114 [ 376 ] {356852a9-4f05-4b02-acb3-fa27df00eede} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./src/QueryPipeline/SizeLimits.cpp:65: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010de4784 4. ./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dac9f4 5. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000010dc5a24 6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000000f7ce184 7. ./contrib/llvm-project/libcxx/include/vector:666: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000000f7cdc14 8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2111: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5c200c 9. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a4508 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a3708 11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f5b80d4 12. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000000f5a423c 13. ./contrib/llvm-project/libcxx/include/vector:434: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000000f59d67c 14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:1201: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x000000000f59b6ac 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:184: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000000f599f24 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) (.llvm.5699878785432179294) @ 0x000000000f7cce24 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x000000000f7cae3c 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) (.llvm.5699878785432179294) @ 0x000000000f7ce908 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000f76fc70 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000000fb3c3a4 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000000fb38df0 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x00000000109ac35c 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 28. ? @ 0x000000000007d5c8 29. ? @ 0x00000000000e5edc 2024.11.06 01:15:25.078558 [ 376 ] {e6526f1f-afb3-4237-a9ad-e18f96851412} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.16 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45456) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:15:25.079029 [ 376 ] {e6526f1f-afb3-4237-a9ad-e18f96851412} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.16 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000000ec04790 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec0464c 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:22.292769 [ 377 ] {ee235897-1879-4734-ba55-29f681f97544} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 585.44 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57822) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000000ec04928 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec044e4 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:22.301755 [ 377 ] {ee235897-1879-4734-ba55-29f681f97544} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 585.44 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000000ec04928 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec044e4 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:22.433122 [ 377 ] {469ad3fc-7257-471b-86ea-51f041018d5d} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 592.89 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57822) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000000ec04928 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec044e4 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:22.440332 [ 377 ] {469ad3fc-7257-471b-86ea-51f041018d5d} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 592.89 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000000ec04928 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec044e4 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:22.516605 [ 377 ] {9066e268-c9f3-4a3c-8790-babf24c9e227} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 576.62 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57822) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000000ec04928 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec044e4 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:22.519522 [ 377 ] {9066e268-c9f3-4a3c-8790-babf24c9e227} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 576.62 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000000ec04928 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000000ec044e4 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x000000000ebf0ea0 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x0000000010a17504 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x0000000010a0d8f0 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x0000000010a0ee7c 7. ./base/glibc-compatibility/musl/clock_gettime.c:62: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000be04318 8. ./src/Common/ThreadPool.h:226: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000be08040 9. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:23.760600 [ 377 ] {db3f9948-7776-45b1-bc25-cc5b0bef90fd} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.125173884 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57834) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:23.835135 [ 377 ] {db3f9948-7776-45b1-bc25-cc5b0bef90fd} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.125173884 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:23.845275 [ 1963 ] {f48ff121-b459-4d57-b99b-73e297ddd6bc} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39752). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39752) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:23.896087 [ 1963 ] {f48ff121-b459-4d57-b99b-73e297ddd6bc} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39752). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:23.896169 [ 1963 ] {f48ff121-b459-4d57-b99b-73e297ddd6bc} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39752). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:23.896351 [ 1963 ] {f48ff121-b459-4d57-b99b-73e297ddd6bc} TCPHandler: Client has gone away. 2024.11.06 01:16:25.011451 [ 377 ] {b0489be0-c6bb-440e-8406-8863b44b1adb} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.10251557 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56430) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:25.075175 [ 377 ] {b0489be0-c6bb-440e-8406-8863b44b1adb} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.10251557 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:25.101414 [ 1963 ] {d32f6a44-9e08-4042-ab0b-4c0c9a2b7338} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51270). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51270) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:25.115585 [ 1964 ] {0afde829-d823-4a8c-a123-072031d66d26} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39520). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39520) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:25.128255 [ 1963 ] {d32f6a44-9e08-4042-ab0b-4c0c9a2b7338} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51270). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:25.128310 [ 1963 ] {d32f6a44-9e08-4042-ab0b-4c0c9a2b7338} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51270). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:25.128477 [ 1963 ] {d32f6a44-9e08-4042-ab0b-4c0c9a2b7338} TCPHandler: Client has gone away. 2024.11.06 01:16:25.141235 [ 1964 ] {0afde829-d823-4a8c-a123-072031d66d26} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39520). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:25.141298 [ 1964 ] {0afde829-d823-4a8c-a123-072031d66d26} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39520). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:25.141455 [ 1964 ] {0afde829-d823-4a8c-a123-072031d66d26} TCPHandler: Client has gone away. 2024.11.06 01:16:26.208860 [ 376 ] {e32ffe57-b35c-4c31-96ec-cd0df72a90f2} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.045143525 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56444) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:26.278210 [ 1963 ] {43d7106a-e834-4388-868e-b905bfc0fc10} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51280). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51280) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:26.283342 [ 376 ] {e32ffe57-b35c-4c31-96ec-cd0df72a90f2} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.045143525 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:26.300296 [ 1965 ] {b99b228c-7ef6-4fa1-a12f-8ffb3120b077} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39532). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39532) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:26.302877 [ 1964 ] {ef8ff273-385a-4122-b68e-955f235e29f6} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56308). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56308) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:26.308577 [ 1963 ] {43d7106a-e834-4388-868e-b905bfc0fc10} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51280). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:26.308639 [ 1963 ] {43d7106a-e834-4388-868e-b905bfc0fc10} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51280). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:26.308773 [ 1963 ] {43d7106a-e834-4388-868e-b905bfc0fc10} TCPHandler: Client has gone away. 2024.11.06 01:16:26.327189 [ 1965 ] {b99b228c-7ef6-4fa1-a12f-8ffb3120b077} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39532). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:26.327271 [ 1965 ] {b99b228c-7ef6-4fa1-a12f-8ffb3120b077} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39532). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:26.327426 [ 1965 ] {b99b228c-7ef6-4fa1-a12f-8ffb3120b077} TCPHandler: Client has gone away. 2024.11.06 01:16:26.340371 [ 1964 ] {ef8ff273-385a-4122-b68e-955f235e29f6} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56308). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:26.340475 [ 1964 ] {ef8ff273-385a-4122-b68e-955f235e29f6} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56308). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:26.340607 [ 1964 ] {ef8ff273-385a-4122-b68e-955f235e29f6} TCPHandler: Client has gone away. 2024.11.06 01:16:27.443338 [ 376 ] {873d03b6-8391-4ebe-a523-ab820b295fb3} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.086687026 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56474) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.459299 [ 1963 ] {ee89be0b-7185-4a77-bb4f-13f1b6b4be68} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56318). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56318) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.477596 [ 377 ] {85f26ff3-e2f6-40b5-bb8b-fdeda22c56ce} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37812). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37812) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.504155 [ 1965 ] {808feecb-13ff-4ed6-ad8a-16e3e8e56ab9} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39546). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39546) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.506359 [ 377 ] {85f26ff3-e2f6-40b5-bb8b-fdeda22c56ce} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37812). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:27.506423 [ 377 ] {85f26ff3-e2f6-40b5-bb8b-fdeda22c56ce} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37812). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.506568 [ 377 ] {85f26ff3-e2f6-40b5-bb8b-fdeda22c56ce} TCPHandler: Client has gone away. 2024.11.06 01:16:27.510059 [ 376 ] {873d03b6-8391-4ebe-a523-ab820b295fb3} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.086687026 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.512412 [ 1963 ] {ee89be0b-7185-4a77-bb4f-13f1b6b4be68} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56318). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:27.513242 [ 1963 ] {ee89be0b-7185-4a77-bb4f-13f1b6b4be68} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56318). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.513450 [ 1963 ] {ee89be0b-7185-4a77-bb4f-13f1b6b4be68} TCPHandler: Client has gone away. 2024.11.06 01:16:27.516777 [ 1964 ] {50a61453-927f-4cfb-a384-2e747fc4354a} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51290). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51290) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.529235 [ 1965 ] {808feecb-13ff-4ed6-ad8a-16e3e8e56ab9} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39546). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:27.529297 [ 1965 ] {808feecb-13ff-4ed6-ad8a-16e3e8e56ab9} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39546). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.529460 [ 1965 ] {808feecb-13ff-4ed6-ad8a-16e3e8e56ab9} TCPHandler: Client has gone away. 2024.11.06 01:16:27.543552 [ 1964 ] {50a61453-927f-4cfb-a384-2e747fc4354a} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51290). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:27.543619 [ 1964 ] {50a61453-927f-4cfb-a384-2e747fc4354a} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51290). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:27.543783 [ 1964 ] {50a61453-927f-4cfb-a384-2e747fc4354a} TCPHandler: Client has gone away. 2024.11.06 01:16:28.644354 [ 377 ] {e2db818b-c930-4bcc-b443-5571d7226329} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.046514948 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56496) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.676731 [ 1965 ] {a32ec152-7bed-472b-bec8-f1ac56cea062} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37816). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37816) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.682666 [ 1965 ] {a32ec152-7bed-472b-bec8-f1ac56cea062} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37816). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:28.682802 [ 1965 ] {a32ec152-7bed-472b-bec8-f1ac56cea062} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37816). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.682948 [ 1965 ] {a32ec152-7bed-472b-bec8-f1ac56cea062} TCPHandler: Client has gone away. 2024.11.06 01:16:28.697752 [ 377 ] {e2db818b-c930-4bcc-b443-5571d7226329} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.046514948 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.701603 [ 1963 ] {fb91f83c-1ea4-4409-98c7-d78195f91099} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52960). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52960) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.709106 [ 376 ] {ef878187-0003-4df0-803d-c8bc27108fc3} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56326). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56326) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.726314 [ 1964 ] {de8347a5-816f-4df6-924f-9bde30236f9c} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39552). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39552) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.728488 [ 1963 ] {fb91f83c-1ea4-4409-98c7-d78195f91099} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52960). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:28.728567 [ 1963 ] {fb91f83c-1ea4-4409-98c7-d78195f91099} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52960). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.728804 [ 1963 ] {fb91f83c-1ea4-4409-98c7-d78195f91099} TCPHandler: Client has gone away. 2024.11.06 01:16:28.736466 [ 376 ] {ef878187-0003-4df0-803d-c8bc27108fc3} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56326). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:28.736539 [ 376 ] {ef878187-0003-4df0-803d-c8bc27108fc3} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56326). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.736701 [ 376 ] {ef878187-0003-4df0-803d-c8bc27108fc3} TCPHandler: Client has gone away. 2024.11.06 01:16:28.752382 [ 1964 ] {de8347a5-816f-4df6-924f-9bde30236f9c} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39552). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:28.752513 [ 1964 ] {de8347a5-816f-4df6-924f-9bde30236f9c} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39552). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:28.752733 [ 1964 ] {de8347a5-816f-4df6-924f-9bde30236f9c} TCPHandler: Client has gone away. 2024.11.06 01:16:29.859417 [ 377 ] {3583e70f-d51c-4efd-918e-0d10d65aadef} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.07352853 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56502) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.884911 [ 1968 ] {c89c0f17-6165-4aa1-8940-4a2f94a88bf1} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39568). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39568) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.894982 [ 1968 ] {c89c0f17-6165-4aa1-8940-4a2f94a88bf1} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39568). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:29.895051 [ 1968 ] {c89c0f17-6165-4aa1-8940-4a2f94a88bf1} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39568). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.895241 [ 1968 ] {c89c0f17-6165-4aa1-8940-4a2f94a88bf1} TCPHandler: Client has gone away. 2024.11.06 01:16:29.915910 [ 1677 ] {6b5e38f7-3f2e-4748-81c8-898e6c177600} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56338). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56338) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.926449 [ 1677 ] {6b5e38f7-3f2e-4748-81c8-898e6c177600} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56338). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:29.926528 [ 1677 ] {6b5e38f7-3f2e-4748-81c8-898e6c177600} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56338). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.926676 [ 1677 ] {6b5e38f7-3f2e-4748-81c8-898e6c177600} TCPHandler: Client has gone away. 2024.11.06 01:16:29.929334 [ 1963 ] {2899bb9e-1ff6-4825-8dd6-3fedb1317e3f} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52970). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52970) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.941259 [ 1965 ] {a7a46d19-cf97-427c-b2bd-baa88f143f24} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51300). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51300) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.946440 [ 377 ] {3583e70f-d51c-4efd-918e-0d10d65aadef} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.07352853 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.953389 [ 1963 ] {2899bb9e-1ff6-4825-8dd6-3fedb1317e3f} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52970). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:29.953457 [ 1963 ] {2899bb9e-1ff6-4825-8dd6-3fedb1317e3f} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52970). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.953596 [ 1963 ] {2899bb9e-1ff6-4825-8dd6-3fedb1317e3f} TCPHandler: Client has gone away. 2024.11.06 01:16:29.955458 [ 1965 ] {a7a46d19-cf97-427c-b2bd-baa88f143f24} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51300). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:29.957766 [ 1965 ] {a7a46d19-cf97-427c-b2bd-baa88f143f24} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51300). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.958018 [ 1965 ] {a7a46d19-cf97-427c-b2bd-baa88f143f24} TCPHandler: Client has gone away. 2024.11.06 01:16:29.958452 [ 1964 ] {096d6b4b-84e2-4bc3-9691-cbd577adf1e3} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:59060). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59060) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.964233 [ 376 ] {475f24cb-ba37-46f6-9d77-488f77f6c599} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37828). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37828) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.979347 [ 1964 ] {096d6b4b-84e2-4bc3-9691-cbd577adf1e3} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:59060). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:29.979419 [ 1964 ] {096d6b4b-84e2-4bc3-9691-cbd577adf1e3} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:59060). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.979560 [ 1964 ] {096d6b4b-84e2-4bc3-9691-cbd577adf1e3} TCPHandler: Client has gone away. 2024.11.06 01:16:29.984405 [ 376 ] {475f24cb-ba37-46f6-9d77-488f77f6c599} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37828). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:29.984503 [ 376 ] {475f24cb-ba37-46f6-9d77-488f77f6c599} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37828). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:29.984708 [ 376 ] {475f24cb-ba37-46f6-9d77-488f77f6c599} TCPHandler: Client has gone away. 2024.11.06 01:16:31.163861 [ 1968 ] {db2c8613-b592-4366-a4f1-5a2c9ac57397} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.133188796 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56506) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.182106 [ 1677 ] {c56fb7fa-b273-4c00-835d-cb3e03d31e87} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39570). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39570) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.198947 [ 1962 ] {ccaeb738-244c-4f5d-a17c-c6f5c4badd94} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:60366). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60366) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.211082 [ 1962 ] {ccaeb738-244c-4f5d-a17c-c6f5c4badd94} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:60366). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:31.211138 [ 1962 ] {ccaeb738-244c-4f5d-a17c-c6f5c4badd94} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:60366). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.211289 [ 1962 ] {ccaeb738-244c-4f5d-a17c-c6f5c4badd94} TCPHandler: Client has gone away. 2024.11.06 01:16:31.217616 [ 1677 ] {c56fb7fa-b273-4c00-835d-cb3e03d31e87} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39570). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:31.217675 [ 1677 ] {c56fb7fa-b273-4c00-835d-cb3e03d31e87} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39570). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.217818 [ 1677 ] {c56fb7fa-b273-4c00-835d-cb3e03d31e87} TCPHandler: Client has gone away. 2024.11.06 01:16:31.220137 [ 3191 ] {e0626d51-b0c0-41ce-8f18-9e2f4b96d39c} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37838). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37838) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.220286 [ 377 ] {1aa77908-2178-4758-b5fd-0472a87579d4} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56340). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56340) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.220297 [ 1586 ] {e68f3da2-c484-45a0-b290-80b5ed34fac6} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:59072). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59072) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.220953 [ 376 ] {bbe80a83-daef-489b-bdea-44393392d00e} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52978). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52978) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.227371 [ 377 ] {1aa77908-2178-4758-b5fd-0472a87579d4} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56340). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:31.227448 [ 377 ] {1aa77908-2178-4758-b5fd-0472a87579d4} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56340). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.227584 [ 376 ] {bbe80a83-daef-489b-bdea-44393392d00e} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52978). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:31.227658 [ 376 ] {bbe80a83-daef-489b-bdea-44393392d00e} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52978). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.227790 [ 376 ] {bbe80a83-daef-489b-bdea-44393392d00e} TCPHandler: Client has gone away. 2024.11.06 01:16:31.227603 [ 377 ] {1aa77908-2178-4758-b5fd-0472a87579d4} TCPHandler: Client has gone away. 2024.11.06 01:16:31.233490 [ 3191 ] {e0626d51-b0c0-41ce-8f18-9e2f4b96d39c} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37838). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:31.233557 [ 3191 ] {e0626d51-b0c0-41ce-8f18-9e2f4b96d39c} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37838). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.233695 [ 3191 ] {e0626d51-b0c0-41ce-8f18-9e2f4b96d39c} TCPHandler: Client has gone away. 2024.11.06 01:16:31.234617 [ 1968 ] {db2c8613-b592-4366-a4f1-5a2c9ac57397} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.133188796 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.235965 [ 1586 ] {e68f3da2-c484-45a0-b290-80b5ed34fac6} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:59072). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:31.236020 [ 1586 ] {e68f3da2-c484-45a0-b290-80b5ed34fac6} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:59072). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.236183 [ 1586 ] {e68f3da2-c484-45a0-b290-80b5ed34fac6} TCPHandler: Client has gone away. 2024.11.06 01:16:31.246898 [ 1585 ] {99ad0f22-3850-419f-a980-5f17495cf3f4} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51314). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51314) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.265324 [ 1585 ] {99ad0f22-3850-419f-a980-5f17495cf3f4} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51314). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:31.265381 [ 1585 ] {99ad0f22-3850-419f-a980-5f17495cf3f4} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51314). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:31.265544 [ 1585 ] {99ad0f22-3850-419f-a980-5f17495cf3f4} TCPHandler: Client has gone away. 2024.11.06 01:16:32.425001 [ 1968 ] {e1b781da-955b-4173-b003-3c0ec1f19561} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.103511187 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56508) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.456364 [ 1962 ] {d3d03944-8c41-4c67-a09c-f488aba913e3} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52994). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52994) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.464906 [ 1586 ] {576c197d-1de5-4676-bf46-4a4630801c6a} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:59074). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59074) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.465169 [ 1968 ] {e1b781da-955b-4173-b003-3c0ec1f19561} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.103511187 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.468749 [ 1962 ] {d3d03944-8c41-4c67-a09c-f488aba913e3} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52994). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:32.468807 [ 1962 ] {d3d03944-8c41-4c67-a09c-f488aba913e3} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52994). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.468931 [ 1963 ] {8fe440c9-1d67-4349-9256-343727d7aa60} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51330). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51330) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.468968 [ 1962 ] {d3d03944-8c41-4c67-a09c-f488aba913e3} TCPHandler: Client has gone away. 2024.11.06 01:16:32.481848 [ 1586 ] {576c197d-1de5-4676-bf46-4a4630801c6a} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:59074). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:32.481920 [ 1586 ] {576c197d-1de5-4676-bf46-4a4630801c6a} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:59074). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.482059 [ 1586 ] {576c197d-1de5-4676-bf46-4a4630801c6a} TCPHandler: Client has gone away. 2024.11.06 01:16:32.485979 [ 1963 ] {8fe440c9-1d67-4349-9256-343727d7aa60} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51330). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:32.486078 [ 1963 ] {8fe440c9-1d67-4349-9256-343727d7aa60} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51330). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.486264 [ 1963 ] {8fe440c9-1d67-4349-9256-343727d7aa60} TCPHandler: Client has gone away. 2024.11.06 01:16:32.488866 [ 377 ] {96fc63d3-9a12-40df-bcff-f540c8f99877} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56348). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56348) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.489217 [ 3191 ] {102a8698-72f4-4934-96ac-2bd8e702c65b} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39584). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39584) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.489812 [ 1677 ] {8b9c5059-4e5d-4f08-b0e2-fccca9f72563} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:60376). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60376) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.489850 [ 1585 ] {1b926c57-5aa0-4675-93e0-1b0f0ae83482} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:35292). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35292) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.490668 [ 376 ] {a41e4eb3-5a14-4680-8205-d76edcaddd84} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37850). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37850) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.506153 [ 377 ] {96fc63d3-9a12-40df-bcff-f540c8f99877} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56348). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:32.506224 [ 377 ] {96fc63d3-9a12-40df-bcff-f540c8f99877} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56348). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.506355 [ 377 ] {96fc63d3-9a12-40df-bcff-f540c8f99877} TCPHandler: Client has gone away. 2024.11.06 01:16:32.511983 [ 376 ] {a41e4eb3-5a14-4680-8205-d76edcaddd84} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37850). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:32.512063 [ 376 ] {a41e4eb3-5a14-4680-8205-d76edcaddd84} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:37850). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.512195 [ 376 ] {a41e4eb3-5a14-4680-8205-d76edcaddd84} TCPHandler: Client has gone away. 2024.11.06 01:16:32.514820 [ 3191 ] {102a8698-72f4-4934-96ac-2bd8e702c65b} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39584). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:32.514895 [ 3191 ] {102a8698-72f4-4934-96ac-2bd8e702c65b} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39584). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x00000000109bc65c 4. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::runImpl() @ 0x00000000109aeedc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.515049 [ 3191 ] {102a8698-72f4-4934-96ac-2bd8e702c65b} TCPHandler: Client has gone away. 2024.11.06 01:16:32.522675 [ 1585 ] {1b926c57-5aa0-4675-93e0-1b0f0ae83482} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:35292). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:32.522732 [ 1677 ] {8b9c5059-4e5d-4f08-b0e2-fccca9f72563} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:60376). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:32.522819 [ 1585 ] {1b926c57-5aa0-4675-93e0-1b0f0ae83482} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:35292). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.522837 [ 1677 ] {8b9c5059-4e5d-4f08-b0e2-fccca9f72563} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:60376). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:32.523017 [ 1585 ] {1b926c57-5aa0-4675-93e0-1b0f0ae83482} TCPHandler: Client has gone away. 2024.11.06 01:16:32.523073 [ 1677 ] {8b9c5059-4e5d-4f08-b0e2-fccca9f72563} TCPHandler: Client has gone away. 2024.11.06 01:16:33.737958 [ 1677 ] {6b83bb3d-63d1-4912-b63e-e52c7b1e23fb} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.032345989 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60378) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.746643 [ 1585 ] {10e8c8a4-90e1-45b1-84c8-ab5f39b48ef9} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.018880193 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59082) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.755242 [ 1968 ] {a0a9e602-2ab1-4901-8219-f290e9020ba1} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.20393196 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56516) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.761939 [ 3191 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:37860). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000be8028c 2. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000be7ff54 3. ./src/IO/ReadBuffer.h:71: DB::TCPHandler::runImpl() @ 0x00000000109ab8bc 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 8. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 9. ? @ 0x000000000007d5c8 10. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:33.766148 [ 376 ] {c03557c8-dec7-47aa-b9a6-b88709e115fb} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51342). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51342) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::runImpl() @ 0x00000000109b0a34 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 8. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 9. ? @ 0x000000000007d5c8 10. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.766193 [ 1964 ] {5bfaa9cd-5df9-4345-bb58-d3e77b60a117} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:53010). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53010) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.766232 [ 1963 ] {c209a181-45ee-4143-a7bc-c8bd87c64a44} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:45356). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45356) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::runImpl() @ 0x00000000109b0a34 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 8. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 9. ? @ 0x000000000007d5c8 10. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.766458 [ 376 ] {c03557c8-dec7-47aa-b9a6-b88709e115fb} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51342). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:33.766502 [ 1677 ] {6b83bb3d-63d1-4912-b63e-e52c7b1e23fb} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.032345989 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.766505 [ 1963 ] {c209a181-45ee-4143-a7bc-c8bd87c64a44} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:45356). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:33.766510 [ 376 ] {c03557c8-dec7-47aa-b9a6-b88709e115fb} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:51342). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::runImpl() @ 0x00000000109b0a34 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 8. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 9. ? @ 0x000000000007d5c8 10. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.772692 [ 1586 ] {cda4a82a-3a7f-420a-8ca1-0faa7793036a} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39586). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39586) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.772765 [ 1963 ] {c209a181-45ee-4143-a7bc-c8bd87c64a44} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:45356). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::runImpl() @ 0x00000000109b0a34 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 8. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 9. ? @ 0x000000000007d5c8 10. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.772837 [ 1585 ] {10e8c8a4-90e1-45b1-84c8-ab5f39b48ef9} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.018880193 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.772877 [ 1677 ] {6b83bb3d-63d1-4912-b63e-e52c7b1e23fb} TCPHandler: Client has gone away. 2024.11.06 01:16:33.772952 [ 376 ] {c03557c8-dec7-47aa-b9a6-b88709e115fb} TCPHandler: Client has gone away. 2024.11.06 01:16:33.773033 [ 1585 ] {10e8c8a4-90e1-45b1-84c8-ab5f39b48ef9} TCPHandler: Client has gone away. 2024.11.06 01:16:33.773185 [ 1963 ] {c209a181-45ee-4143-a7bc-c8bd87c64a44} TCPHandler: Client has gone away. 2024.11.06 01:16:33.774008 [ 1962 ] {292069d4-525b-4230-b894-98df90f1d2a2} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56356). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56356) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::runImpl() @ 0x00000000109b0a34 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 8. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 9. ? @ 0x000000000007d5c8 10. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.774292 [ 1962 ] {292069d4-525b-4230-b894-98df90f1d2a2} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56356). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:33.774363 [ 1962 ] {292069d4-525b-4230-b894-98df90f1d2a2} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56356). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::runImpl() @ 0x00000000109b0a34 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 8. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 9. ? @ 0x000000000007d5c8 10. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.774516 [ 1962 ] {292069d4-525b-4230-b894-98df90f1d2a2} TCPHandler: Client has gone away. 2024.11.06 01:16:33.777190 [ 377 ] {1a1ceac5-7827-408e-8144-c6184c100435} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:35308). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:35308) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::runImpl() @ 0x00000000109b0a34 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 8. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 9. ? @ 0x000000000007d5c8 10. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.777497 [ 377 ] {1a1ceac5-7827-408e-8144-c6184c100435} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:35308). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:33.777541 [ 377 ] {1a1ceac5-7827-408e-8144-c6184c100435} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:35308). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::runImpl() @ 0x00000000109b0a34 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 8. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 9. ? @ 0x000000000007d5c8 10. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.777686 [ 377 ] {1a1ceac5-7827-408e-8144-c6184c100435} TCPHandler: Client has gone away. 2024.11.06 01:16:33.778408 [ 1964 ] {5bfaa9cd-5df9-4345-bb58-d3e77b60a117} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:53010). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:33.778481 [ 1964 ] {5bfaa9cd-5df9-4345-bb58-d3e77b60a117} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:53010). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.778672 [ 1964 ] {5bfaa9cd-5df9-4345-bb58-d3e77b60a117} TCPHandler: Client has gone away. 2024.11.06 01:16:33.799286 [ 1968 ] {a0a9e602-2ab1-4901-8219-f290e9020ba1} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.20393196 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000000ebf1ef8 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000000ebf1e1c 3. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x0000000010a0d2a0 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000010a0d028 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000010a1a690 6. ./base/base/../base/wide_integer_impl.h:810: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000be06dac 7. ? @ 0x000000000007d5c8 8. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.805844 [ 1586 ] {cda4a82a-3a7f-420a-8ca1-0faa7793036a} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39586). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81f68 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogs() @ 0x00000000109bb0a8 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::runImpl() @ 0x00000000109b1008 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc (version 24.3.12.76.altinitystable (altinity build)) 2024.11.06 01:16:33.805922 [ 1586 ] {cda4a82a-3a7f-420a-8ca1-0faa7793036a} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:39586). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bd69808 1. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000be816e0 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000be81e18 3. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x00000000109ba330 4. ./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000109af1d4 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:594: DB::TCPHandler::run() @ 0x00000000109c31a8 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000012d557c4 7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000012d56790 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000012e76494 9. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000012e7484c 10. ? @ 0x000000000007d5c8 11. ? @ 0x00000000000e5edc 2024.11.06 01:16:33.806051 [ 1586 ] {cda4a82a-3a7f-420a-8ca1-0faa7793036a} TCPHandler: Client has gone away. 2024.11.06 01:28:03.914634 [ 373 ] {} Application: Closed all listening sockets. Waiting for 4 outstanding connections.