2025.04.06 20:01:40.511113 [ 344 ] {} Context: Server was built with undefined behavior sanitizer. It will work slowly. 2025.04.06 20:01:40.933795 [ 344 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2025.04.06 20:01:40.933974 [ 344 ] {} KeeperLogStore: Removing all changelogs 2025.04.06 20:01:40.934245 [ 344 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2025.04.06 20:01:40.935672 [ 344 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2025.04.06 20:01:40.935681 [ 344 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2025.04.06 20:01:40.986102 [ 929 ] {} RaftInstance: Election timeout, initiate leader election 2025.04.06 20:01:41.007714 [ 344 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2025.04.06 20:01:41.007748 [ 344 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2025.04.06 20:01:41.015309 [ 344 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2025.04.06 20:01:41.015911 [ 344 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2025.04.06 20:01:42.203093 [ 348 ] {d151c900-ce3e-4942-9559-a97f5260d733} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2025.04.06 20:01:52.512402 [ 348 ] {351b6107-25e3-4abf-bead-7fdf05006766} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2025.04.06 20:02:17.361502 [ 961 ] {02856a88-5cca-493f-b4eb-97ab3f3b0241::all_1_7_1} virtual bool DB::MergePlainMergeTreeTask::executeStep(): Exception is in merge_task.: Code: 236. DB::Exception: Cancelled merging parts. (ABORTED), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000018f02daf 5. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:263: DB::MergeTask::GlobalRuntimeContext::checkOperationIsNotCanceled() const @ 0x00000000347c8a77 6. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:931: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::finalize() const @ 0x00000000347e4ad1 7. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:892: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() const @ 0x00000000347e4123 8. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:720: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000347e0ff3 9. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1504: DB::MergeTask::execute() @ 0x00000000347ef78e 10. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:57: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000034e04831 11. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:308: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x0000000034830f5d 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:353: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x0000000034831c94 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 14. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 16. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 17. ? @ 0x00007f15d9059ac3 18. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:14.427161 [ 348 ] {2a2fda72-fdc6-4474-a8e4-98257d915f24} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 575.15 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:33190) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000003086780f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:43: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000308671ee 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:14.428477 [ 348 ] {} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 575.15 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000003086780f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:43: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000308671ee 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:14.699233 [ 348 ] {dba528fc-93e1-4aa6-9689-d4f362410ac6} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.25 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:33190) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000003086780f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:43: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000308671ee 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:14.701628 [ 348 ] {} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.25 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000003086780f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:43: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000308671ee 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:14.951366 [ 348 ] {d6481fa9-a478-4357-a3de-c622e9a2e1b3} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 575.18 MiB: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:33190) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000003086780f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:43: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000308671ee 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:14.952333 [ 348 ] {} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 575.18 MiB: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000003086780f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:43: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000308671ee 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:47.725904 [ 1747 ] {15e0833d-c9fa-4ad4-9b55-9ad10515cf19} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58354) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:315: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035c9b570 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2082: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cbe435 8. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c4c55c 9. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c84ab9 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:261: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003223b8d3 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:228: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003223b4c6 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:604: DB::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310af736 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3854: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000310a56fe 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3975: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000310a3dff 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:2879: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310c999a 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3817: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000310a5204 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3975: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000310a3dff 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4299: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310ebe8c 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5563: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000003109a5da 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:172: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x00000000310982eb 21. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:18: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000031097971 22. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:184: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x0000000031c9b8d9 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:149: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000032239e50 24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:167: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000032237837 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000003223c5ce 26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:279: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x000000003223c085 27. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000032188378 28. ./build_docker/./src/Interpreters/executeQuery.cpp:1305: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003273693e 29. ./build_docker/./src/Interpreters/executeQuery.cpp:1535: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003273160e 30. ./build_docker/./src/Server/TCPHandler.cpp:656: DB::TCPHandler::runImpl() @ 0x00000000351ffdbc 31. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 2025.04.06 20:03:47.726401 [ 1747 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:315: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035c9b570 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2082: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cbe435 8. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c4c55c 9. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c84ab9 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:261: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003223b8d3 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:228: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003223b4c6 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:604: DB::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310af736 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3854: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000310a56fe 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3975: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000310a3dff 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:2879: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310c999a 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3817: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000310a5204 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3975: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000310a3dff 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4299: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310ebe8c 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5563: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000003109a5da 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:172: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x00000000310982eb 21. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:18: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000031097971 22. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:184: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x0000000031c9b8d9 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:149: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000032239e50 24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:167: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000032237837 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000003223c5ce 26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:279: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x000000003223c085 27. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000032188378 28. ./build_docker/./src/Interpreters/executeQuery.cpp:1305: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003273693e 29. ./build_docker/./src/Interpreters/executeQuery.cpp:1535: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003273160e 30. ./build_docker/./src/Server/TCPHandler.cpp:656: DB::TCPHandler::runImpl() @ 0x00000000351ffdbc 31. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 2025.04.06 20:03:47.762477 [ 1747 ] {5c155c80-c90f-4930-a338-6ec3cc88e1fc} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58354) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:315: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035c9b570 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2082: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cbe435 8. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c4c55c 9. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c84ab9 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:261: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003223b8d3 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:228: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003223b4c6 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1368: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000032737703 13. ./build_docker/./src/Interpreters/executeQuery.cpp:1535: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003273160e 14. ./build_docker/./src/Server/TCPHandler.cpp:656: DB::TCPHandler::runImpl() @ 0x00000000351ffdbc 15. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 17. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 19. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 20. ? @ 0x00007f15d9059ac3 21. ? @ 0x00007f15d90eb850 2025.04.06 20:03:47.762708 [ 1747 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:315: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035c9b570 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2082: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cbe435 8. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c4c55c 9. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c84ab9 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:261: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003223b8d3 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:228: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003223b4c6 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1368: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000032737703 13. ./build_docker/./src/Interpreters/executeQuery.cpp:1535: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003273160e 14. ./build_docker/./src/Server/TCPHandler.cpp:656: DB::TCPHandler::runImpl() @ 0x00000000351ffdbc 15. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 17. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 19. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 20. ? @ 0x00007f15d9059ac3 21. ? @ 0x00007f15d90eb850 2025.04.06 20:03:47.858948 [ 1747 ] {c2bde285-3d0a-4a6c-8ee1-fba2ce99162e} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.79 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58354) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:47.860834 [ 1747 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.79 million: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:47.904459 [ 1747 ] {a862f2a8-17e2-47b1-a3d8-8173763e85ed} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58354) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:315: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035c9b570 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2082: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cbe435 8. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c4c55c 9. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c84ab9 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:261: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003223b8d3 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:228: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003223b4c6 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:604: DB::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310af736 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3854: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000310a56fe 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3975: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000310a3dff 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:2879: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310c999a 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3817: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000310a5204 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5588: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000003109a85b 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:172: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x00000000310982eb 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:18: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000031097971 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:184: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x0000000031c9b8d9 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:149: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000032239e50 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:167: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000032237837 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000003223c5ce 24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:279: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x000000003223c085 25. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000032188378 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1305: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003273693e 27. ./build_docker/./src/Interpreters/executeQuery.cpp:1535: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003273160e 28. ./build_docker/./src/Server/TCPHandler.cpp:656: DB::TCPHandler::runImpl() @ 0x00000000351ffdbc 29. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 31. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 2025.04.06 20:03:47.904700 [ 1747 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:315: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035c9b570 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2082: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cbe435 8. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c4c55c 9. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c84ab9 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:261: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000003223b8d3 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:228: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x000000003223b4c6 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:604: DB::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310af736 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3854: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000310a56fe 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3975: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000310a3dff 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:2879: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000310c999a 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3817: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000310a5204 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5588: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000003109a85b 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:172: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x00000000310982eb 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:18: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000031097971 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:184: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x0000000031c9b8d9 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:149: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000032239e50 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:167: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000032237837 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x000000003223c5ce 24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:279: std::unique_ptr> std::__function::__policy_invoker> (DB::InterpreterFactory::Arguments const&)>::__call_impl> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x000000003223c085 25. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000032188378 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1305: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003273693e 27. ./build_docker/./src/Interpreters/executeQuery.cpp:1535: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000003273160e 28. ./build_docker/./src/Server/TCPHandler.cpp:656: DB::TCPHandler::runImpl() @ 0x00000000351ffdbc 29. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 31. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 2025.04.06 20:03:47.923383 [ 1747 ] {2c1d5b7b-a2e3-4272-a0fd-2be44ad56232} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58354) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:47.924020 [ 1747 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:49.132244 [ 955 ] {07eec6fa-bae3-45c2-a38e-550b01e29869::all_1_8_1} virtual bool DB::MergePlainMergeTreeTask::executeStep(): Exception is in merge_task.: Code: 236. DB::Exception: Cancelled merging parts. (ABORTED), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000018f02daf 5. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:263: DB::MergeTask::GlobalRuntimeContext::checkOperationIsNotCanceled() const @ 0x00000000347c8a77 6. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:931: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::finalize() const @ 0x00000000347e4ad1 7. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:892: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() const @ 0x00000000347e4123 8. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:720: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000347e0ff3 9. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1504: DB::MergeTask::execute() @ 0x00000000347ef78e 10. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:57: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000034e04831 11. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:308: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x0000000034830f5d 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:353: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x0000000034831c94 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 14. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 16. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 17. ? @ 0x00007f15d9059ac3 18. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:53.937369 [ 1764 ] {7384abf2-177e-45fc-946d-c92f130284aa} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.67 million: While executing Remote. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58418) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:53.938815 [ 1764 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.67 million: While executing Remote. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:53.973672 [ 1737 ] {52299fcc-954c-4574-9d0a-e2c247bc9ff4} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41186). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:53.973710 [ 1761 ] {837fe396-c992-4d2e-b967-d61a0e0898ec} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56888). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:53.973676 [ 1760 ] {3df5893a-1b93-45ed-b0e6-914b2d5abf57} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60828). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:53.974366 [ 1761 ] {837fe396-c992-4d2e-b967-d61a0e0898ec} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56888). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:56888) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:53.974552 [ 1760 ] {3df5893a-1b93-45ed-b0e6-914b2d5abf57} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60828). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:60828) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:53.974594 [ 1737 ] {52299fcc-954c-4574-9d0a-e2c247bc9ff4} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41186). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41186) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:53.975949 [ 1760 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60828). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:53.976201 [ 1761 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56888). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:53.977042 [ 1737 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41186). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.233634 [ 1764 ] {59927e3a-ffa6-4fb4-8c52-399fec531a2c} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58432) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.234292 [ 1812 ] {621dd5d8-4c5c-43df-8fee-d49fdb12f1ee} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57354). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.234462 [ 1761 ] {acbe6d8b-8bae-49b1-ad68-2006c8696c7c} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41202). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.234719 [ 1760 ] {b4298b79-095f-4ee2-9646-921af95be02a} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56894). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.234760 [ 1812 ] {621dd5d8-4c5c-43df-8fee-d49fdb12f1ee} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57354). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:57354) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.235030 [ 1761 ] {acbe6d8b-8bae-49b1-ad68-2006c8696c7c} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41202). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41202) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.235159 [ 1760 ] {b4298b79-095f-4ee2-9646-921af95be02a} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56894). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:56894) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.235511 [ 1764 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.235728 [ 1812 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57354). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.235964 [ 1761 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41202). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.236161 [ 1760 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56894). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.283000 [ 1737 ] {85eb4b71-bb60-420b-9448-bd6c037e56d7} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60834). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.283577 [ 1737 ] {85eb4b71-bb60-420b-9448-bd6c037e56d7} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60834). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:60834) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.285463 [ 1737 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60834). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.555205 [ 1764 ] {432cfd82-fb95-4b87-88a2-fd1871d71827} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.66 million: While executing Remote. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58434) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.555727 [ 1768 ] {ccd3fb7a-ab2e-4e24-8e8d-2264baa049bb} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41406). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.555945 [ 1812 ] {b153f826-1630-4aa2-b2a6-2b528e5c0d04} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57368). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.556255 [ 1768 ] {ccd3fb7a-ab2e-4e24-8e8d-2264baa049bb} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41406). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41406) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.556463 [ 1812 ] {b153f826-1630-4aa2-b2a6-2b528e5c0d04} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57368). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:57368) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.556943 [ 1764 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.66 million: While executing Remote. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.557338 [ 1768 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41406). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.557502 [ 1812 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57368). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.622033 [ 1760 ] {cd966216-6ecd-45a7-96f5-cc19797c507f} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56902). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.622625 [ 1760 ] {cd966216-6ecd-45a7-96f5-cc19797c507f} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56902). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:56902) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.624499 [ 1760 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56902). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.641229 [ 1761 ] {913fbba4-fa2d-4236-889f-bddbf1dba2a6} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41218). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.641725 [ 1737 ] {c3677896-c890-46a4-82d8-1e2100ad440a} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60836). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.641869 [ 1761 ] {913fbba4-fa2d-4236-889f-bddbf1dba2a6} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41218). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41218) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.642314 [ 1737 ] {c3677896-c890-46a4-82d8-1e2100ad440a} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60836). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:60836) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.643542 [ 1761 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41218). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.644052 [ 1737 ] {} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60836). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.895800 [ 1746 ] {39e46ed5-1a2d-486c-929d-f0ec1b2eed42} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58442) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.896313 [ 1761 ] {bfd6ce28-8895-43f8-b589-7aaf3eadefb3} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52110). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.896987 [ 1745 ] {9a3ad820-acb0-4d0c-804d-62a745712375} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60846). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.897003 [ 1761 ] {bfd6ce28-8895-43f8-b589-7aaf3eadefb3} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52110). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:52110) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.897506 [ 1746 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.897691 [ 1764 ] {e73311ed-b8a3-448e-b38b-6b2e31852d76} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41420). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.898108 [ 1761 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:52110). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.898221 [ 1764 ] {e73311ed-b8a3-448e-b38b-6b2e31852d76} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41420). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41420) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.899156 [ 1764 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41420). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.912540 [ 1745 ] {9a3ad820-acb0-4d0c-804d-62a745712375} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60846). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:60846) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.913438 [ 1745 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60846). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.941898 [ 1760 ] {92f904af-e3f0-4d78-b953-557e1b8fd241} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57378). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.942463 [ 1760 ] {92f904af-e3f0-4d78-b953-557e1b8fd241} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57378). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:57378) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.944217 [ 1760 ] {} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57378). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.948206 [ 1766 ] {4c2ff578-8cc6-4d3f-8f20-b1b3d2754d16} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41232). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:54.948705 [ 1766 ] {4c2ff578-8cc6-4d3f-8f20-b1b3d2754d16} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41232). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41232) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:54.950460 [ 1766 ] {} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41232). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.291673 [ 1738 ] {ff2a4683-1a9a-44ff-96ea-950549feb113} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58458) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.293348 [ 1738 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.295153 [ 1766 ] {3367f333-6359-4277-acbb-1e34bb041bb3} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57382). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:55.295638 [ 1766 ] {3367f333-6359-4277-acbb-1e34bb041bb3} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57382). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:57382) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.296594 [ 1766 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57382). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.304507 [ 1747 ] {} TCPHandler: TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:56910, local: [::ffff:127.0.0.1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x0000000025d5d6f8 5. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x0000000025d5e355 6. DB::ReadBuffer::next() @ 0x00000000190bdc92 7. ./src/IO/ReadBuffer.h:103: DB::TCPHandler::runImpl() @ 0x00000000351ff13c 8. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 10. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 12. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 13. ? @ 0x00007f15d9059ac3 14. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:55.304559 [ 1747 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:56910, local: [::ffff:127.0.0.1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x0000000025d5d6f8 5. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x0000000025d5e355 6. DB::ReadBuffer::next() @ 0x00000000190bdc92 7. ./src/IO/ReadBuffer.h:103: DB::TCPHandler::runImpl() @ 0x00000000351ff13c 8. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 10. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 12. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 13. ? @ 0x00007f15d9059ac3 14. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:55.306171 [ 1745 ] {40b398aa-2aaa-45d5-9fb7-c0b40ff0dfb4} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60860). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:55.306688 [ 1745 ] {40b398aa-2aaa-45d5-9fb7-c0b40ff0dfb4} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60860). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:60860) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.307702 [ 1745 ] {} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60860). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.313202 [ 1764 ] {37138b10-cd2c-4ebd-a48a-8ff1843dfccd} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41238). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41238) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:1470: DB::TCPHandler::sendProfileEvents(DB::QueryState&) @ 0x000000003522680f 7. ./build_docker/./src/Server/TCPHandler.cpp:1485: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x000000003521307e 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.314306 [ 1764 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41238). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:1470: DB::TCPHandler::sendProfileEvents(DB::QueryState&) @ 0x000000003522680f 7. ./build_docker/./src/Server/TCPHandler.cpp:1485: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x000000003521307e 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.338853 [ 1761 ] {fb50f77e-2d93-4c5a-98f9-7df0c9d173e0} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41428). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:55.339302 [ 1761 ] {fb50f77e-2d93-4c5a-98f9-7df0c9d173e0} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41428). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41428) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.340797 [ 1761 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41428). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.665199 [ 1768 ] {34691b4f-c33d-4131-b1fe-e9f257efbef8} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58466) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.666417 [ 1738 ] {a783ef37-5e65-4778-8a95-e94f777486e9} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41432). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:55.666767 [ 1762 ] {96e9c535-3300-4d6a-b13c-a3b1a648ca75} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56912). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:55.666971 [ 1738 ] {a783ef37-5e65-4778-8a95-e94f777486e9} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41432). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41432) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.667194 [ 1762 ] {96e9c535-3300-4d6a-b13c-a3b1a648ca75} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56912). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:56912) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.667313 [ 1768 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.667475 [ 1766 ] {5d567e66-d9f2-46d2-b56b-164a30749b42} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57384). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:55.667903 [ 1738 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41432). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.667958 [ 1766 ] {5d567e66-d9f2-46d2-b56b-164a30749b42} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57384). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:57384) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.668028 [ 1762 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56912). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.669047 [ 1766 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57384). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.670428 [ 1747 ] {7a38b1a3-7ac1-4aaa-a3e4-f3f109112a60} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:44298). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:55.670978 [ 1747 ] {7a38b1a3-7ac1-4aaa-a3e4-f3f109112a60} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:44298). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:44298) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.672004 [ 1747 ] {} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:44298). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:55.703646 [ 1765 ] {30df375e-4e32-4ff4-9c7e-726624cbe0e5} TCPHandler: Can't send logs or exception to client. Close connection.: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60876). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:825: DB::TCPHandler::runImpl() @ 0x00000000352006ea 7. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 9. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 11. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 12. ? @ 0x00007f15d9059ac3 13. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.051633 [ 1760 ] {291ad2f9-21b3-456e-bc8b-23d8e8ae6e95} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58482) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.052785 [ 1746 ] {93c05c48-d7ba-4201-b12f-545d8b950a4d} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41436). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.053274 [ 1768 ] {} TCPHandler: TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:56922, local: [::ffff:127.0.0.1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x0000000025d5d6f8 5. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x0000000025d5e355 6. DB::ReadBuffer::next() @ 0x00000000190bdc92 7. ./src/IO/ReadBuffer.h:103: DB::TCPHandler::runImpl() @ 0x00000000351ff13c 8. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 10. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 12. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 13. ? @ 0x00007f15d9059ac3 14. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.053319 [ 1746 ] {93c05c48-d7ba-4201-b12f-545d8b950a4d} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41436). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41436) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.053861 [ 1760 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.054275 [ 1746 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:41436). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.053325 [ 1768 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:56922, local: [::ffff:127.0.0.1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x0000000025d5d6f8 5. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x0000000025d5e355 6. DB::ReadBuffer::next() @ 0x00000000190bdc92 7. ./src/IO/ReadBuffer.h:103: DB::TCPHandler::runImpl() @ 0x00000000351ff13c 8. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 10. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 12. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 13. ? @ 0x00007f15d9059ac3 14. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.064286 [ 1761 ] {1f9208a9-c6bf-4d07-a60c-132c9fe9a6c5} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41250). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.064858 [ 1761 ] {1f9208a9-c6bf-4d07-a60c-132c9fe9a6c5} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41250). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:41250) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.066114 [ 1761 ] {} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41250). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.080429 [ 1812 ] {d032ea73-9338-4008-bfe1-d21450c887cc} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60878). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.080835 [ 1812 ] {d032ea73-9338-4008-bfe1-d21450c887cc} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60878). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:60878) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.081735 [ 1812 ] {} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60878). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.448868 [ 1737 ] {cdb6d283-a2a0-4561-896c-415378b18433} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::1]:58484) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.450981 [ 1737 ] {} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 20.00 million, current rows: 26.64 million: While executing Remote. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000018eeb8b6 4. ./src/Common/Exception.h:123: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000003086776f 5. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:32: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000030867109 6. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000030824b5b 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:64: DB::ExecutionThreadContext::executeTask() @ 0x00000000353611cf 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:290: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000003534d2fc 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:256: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000003534e9c9 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c7c34f 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c84334 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000025c79b81 13. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:359: ? @ 0x0000000025c80b2a 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.456785 [ 1765 ] {d7380990-705f-4a7d-a80f-1c75b00ddd91} TCPHandler: Can't send logs or exception to client. Close connection.: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:44316). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:825: DB::TCPHandler::runImpl() @ 0x00000000352006ea 7. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 9. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 11. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 12. ? @ 0x00007f15d9059ac3 13. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.466767 [ 1760 ] {9cad7e42-d8a9-4200-997f-d43b46bcb274} TCPHandler: Can't send logs or exception to client. Close connection.: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56930). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:825: DB::TCPHandler::runImpl() @ 0x00000000352006ea 7. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 9. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 11. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 12. ? @ 0x00007f15d9059ac3 13. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.479080 [ 1768 ] {6c47efd3-ab29-43e6-863c-0e0c9fe99cac} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57410). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.480228 [ 1768 ] {6c47efd3-ab29-43e6-863c-0e0c9fe99cac} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57410). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:57410) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.481254 [ 1768 ] {} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:57410). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.490411 [ 1762 ] {62fa471a-2578-47ac-8ae8-4639dd178801} TCPHandler: Can't send logs or exception to client. Close connection.: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:41262). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:160: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d62031 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:825: DB::TCPHandler::runImpl() @ 0x00000000352006ea 7. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 9. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 11. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 12. ? @ 0x00007f15d9059ac3 13. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.500143 [ 1745 ] {e2b0e35d-4d7e-4d78-9a1a-dbf2e8b11d52} void DB::TCPHandler::sendData(QueryState &, const Block &): Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:57872). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 (version 24.12.2.20234.altinityantalya (altinity build)) 2025.04.06 20:03:56.500660 [ 1745 ] {e2b0e35d-4d7e-4d78-9a1a-dbf2e8b11d52} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:57872). (NETWORK_ERROR) (version 24.12.2.20234.altinityantalya (altinity build)) (from [::ffff:127.0.0.1]:57872) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:03:56.501787 [ 1745 ] {} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:57872). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. std::exception::exception[abi:v15007]() @ 0x000000001904dbf4 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x0000000038c64780 2. ./build_docker/./src/Common/Exception.cpp:105: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000025b5852b 3. ./src/Common/Exception.h:105: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000025d5f107 4. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:171: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000025d623d1 5. DB::WriteBuffer::next() @ 0x0000000018f046a3 6. ./build_docker/./src/Server/TCPHandler.cpp:2464: DB::TCPHandler::sendData(DB::QueryState&, DB::Block const&) @ 0x0000000035222b83 7. ./build_docker/./src/Server/TCPHandler.cpp:1260: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x00000000352130f2 8. ./build_docker/./src/Server/TCPHandler.cpp:670: DB::TCPHandler::runImpl() @ 0x00000000351ffff9 9. ./build_docker/./src/Server/TCPHandler.cpp:2603: DB::TCPHandler::run() @ 0x000000003522af2e 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000038d97e02 11. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000038d98ad2 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x0000000038cfb457 13. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000038cf6d4e 14. ? @ 0x00007f15d9059ac3 15. ? @ 0x00007f15d90eb850 2025.04.06 20:07:30.703415 [ 344 ] {} Application: Closed all listening sockets. Waiting for 4 outstanding connections. 2025.04.06 20:07:35.610339 [ 344 ] {} Application: Closed connections. But 4 remain. Tip: To increase wait time add to config: 60 2025.04.06 20:07:35.610415 [ 344 ] {} Application: Will shutdown forcefully.