2024.10.22 05:22:37.744046 [ 339 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.10.22 05:22:37.906348 [ 339 ] {} Context: The setting 'merge_tree_metadata_cache' is enabled. But the feature of 'metadata cache in RocksDB' is experimental and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. See the corresponding report at https://github.com/ClickHouse/ClickHouse/issues/51182 2024.10.22 05:22:37.989922 [ 339 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2024.10.22 05:22:37.990127 [ 339 ] {} KeeperLogStore: Removing all changelogs 2024.10.22 05:22:37.990392 [ 339 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.10.22 05:22:37.991204 [ 339 ] {} KeeperStateManager: No state was read 2024.10.22 05:22:37.991214 [ 339 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.10.22 05:22:37.991220 [ 339 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.10.22 05:22:38.041793 [ 551 ] {} RaftInstance: Election timeout, initiate leader election 2024.10.22 05:22:38.053414 [ 339 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2024.10.22 05:22:38.053437 [ 339 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2024.10.22 05:22:38.075940 [ 339 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.10.22 05:23:13.571854 [ 339 ] {} KeeperLogStore: Changelog is shut down 2024.10.22 05:23:14.176230 [ 722 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.10.22 05:23:14.349939 [ 722 ] {} Context: The setting 'merge_tree_metadata_cache' is enabled. But the feature of 'metadata cache in RocksDB' is experimental and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. See the corresponding report at https://github.com/ClickHouse/ClickHouse/issues/51182 2024.10.22 05:23:14.445861 [ 722 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.10.22 05:23:14.446965 [ 722 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.10.22 05:23:14.446974 [ 722 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.10.22 05:23:14.497477 [ 934 ] {} RaftInstance: Election timeout, initiate leader election 2024.10.22 05:23:14.629423 [ 722 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.10.22 05:24:00.362143 [ 725 ] {aa86cb17-9cf8-4c95-928a-75ea9505356d} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery1: While processing 1 = ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery1). (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:51090) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135c8f66 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000013589dac in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135a1b2a in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x0000000011f8b6d9 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:188: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x00000000120118d7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::InDepthNodeVisitor>::doVisit(std::shared_ptr&) @ 0x00000000120101fd in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010359 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ./build_docker/./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000001201504a in /usr/lib/debug/usr/bin/clickhouse.debug 11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor>::doVisit(std::shared_ptr&) @ 0x000000001201021e in /usr/lib/debug/usr/bin/clickhouse.debug 12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010359 in /usr/lib/debug/usr/bin/clickhouse.debug 13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010364 in /usr/lib/debug/usr/bin/clickhouse.debug 14. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1551: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr&, std::shared_ptr, unsigned long, std::map, std::allocator>>&, std::map, std::allocator>>&, bool, bool) @ 0x0000000012233991 in /usr/lib/debug/usr/bin/clickhouse.debug 15. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyzeSelect(std::shared_ptr&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector> const&, std::vector> const&, std::shared_ptr) const @ 0x000000001222f501 in /usr/lib/debug/usr/bin/clickhouse.debug 16. ./build_docker/./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr const&, std::shared_ptr const&, std::optional, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&, std::shared_ptr const&, std::shared_ptr)::$_0::operator()(bool) const @ 0x0000000011ee283c in /usr/lib/debug/usr/bin/clickhouse.debug 17. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr const&, std::shared_ptr const&, std::optional, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000011ed6895 in /usr/lib/debug/usr/bin/clickhouse.debug 18. ./build_docker/./contrib/llvm-project/libcxx/include/optional:260: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr const&, std::shared_ptr, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000011f86628 in /usr/lib/debug/usr/bin/clickhouse.debug 19. ./build_docker/./contrib/llvm-project/libcxx/include/vector:434: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000011e8c83e in /usr/lib/debug/usr/bin/clickhouse.debug 20. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122d12aa in /usr/lib/debug/usr/bin/clickhouse.debug 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122cd295 in /usr/lib/debug/usr/bin/clickhouse.debug 22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000131543b9 in /usr/lib/debug/usr/bin/clickhouse.debug 23. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 25. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 27. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 28. ? @ 0x00007ffaf603cac3 in ? 29. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:00.362367 [ 725 ] {aa86cb17-9cf8-4c95-928a-75ea9505356d} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery1: While processing 1 = ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery1). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135c8f66 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000013589dac in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135a1b2a in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x0000000011f8b6d9 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:188: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x00000000120118d7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::InDepthNodeVisitor>::doVisit(std::shared_ptr&) @ 0x00000000120101fd in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010359 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ./build_docker/./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000001201504a in /usr/lib/debug/usr/bin/clickhouse.debug 11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor>::doVisit(std::shared_ptr&) @ 0x000000001201021e in /usr/lib/debug/usr/bin/clickhouse.debug 12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010359 in /usr/lib/debug/usr/bin/clickhouse.debug 13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010364 in /usr/lib/debug/usr/bin/clickhouse.debug 14. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1551: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr&, std::shared_ptr, unsigned long, std::map, std::allocator>>&, std::map, std::allocator>>&, bool, bool) @ 0x0000000012233991 in /usr/lib/debug/usr/bin/clickhouse.debug 15. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyzeSelect(std::shared_ptr&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector> const&, std::vector> const&, std::shared_ptr) const @ 0x000000001222f501 in /usr/lib/debug/usr/bin/clickhouse.debug 16. ./build_docker/./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr const&, std::shared_ptr const&, std::optional, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&, std::shared_ptr const&, std::shared_ptr)::$_0::operator()(bool) const @ 0x0000000011ee283c in /usr/lib/debug/usr/bin/clickhouse.debug 17. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr const&, std::shared_ptr const&, std::optional, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000011ed6895 in /usr/lib/debug/usr/bin/clickhouse.debug 18. ./build_docker/./contrib/llvm-project/libcxx/include/optional:260: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr const&, std::shared_ptr, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000011f86628 in /usr/lib/debug/usr/bin/clickhouse.debug 19. ./build_docker/./contrib/llvm-project/libcxx/include/vector:434: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000011e8c83e in /usr/lib/debug/usr/bin/clickhouse.debug 20. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122d12aa in /usr/lib/debug/usr/bin/clickhouse.debug 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122cd295 in /usr/lib/debug/usr/bin/clickhouse.debug 22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000131543b9 in /usr/lib/debug/usr/bin/clickhouse.debug 23. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 25. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 27. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 28. ? @ 0x00007ffaf603cac3 in ? 29. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:00.371304 [ 725 ] {f3150a7d-d956-412c-94c4-dc1e2b8e48dc} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:51090) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135c8f66 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000013589dac in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135a1b2a in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x0000000011f8b6d9 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122d1b28 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122cd295 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000131543b9 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 12. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 14. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 15. ? @ 0x00007ffaf603cac3 in ? 16. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:00.371440 [ 725 ] {f3150a7d-d956-412c-94c4-dc1e2b8e48dc} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135c8f66 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000013589dac in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135a1b2a in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x0000000011f8b6d9 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122d1b28 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122cd295 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000131543b9 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 12. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 14. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 15. ? @ 0x00007ffaf603cac3 in ? 16. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:00.397255 [ 725 ] {3f44163c-b931-44c4-8afe-6fb36e18ba82} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeThread. (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:51090) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:00.401469 [ 725 ] {3f44163c-b931-44c4-8afe-6fb36e18ba82} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeThread. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:00.418187 [ 725 ] {877e8b14-111a-4a54-9df2-4f246262ed4c} executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery4: While processing CounterID > ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery4). (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:51090) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135c8f66 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000013589dac in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135a1b2a in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x0000000011f8b6d9 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:188: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x00000000120118d7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::InDepthNodeVisitor>::doVisit(std::shared_ptr&) @ 0x00000000120101fd in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010359 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ./build_docker/./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000001201504a in /usr/lib/debug/usr/bin/clickhouse.debug 11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor>::doVisit(std::shared_ptr&) @ 0x000000001201021e in /usr/lib/debug/usr/bin/clickhouse.debug 12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010359 in /usr/lib/debug/usr/bin/clickhouse.debug 13. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1551: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr&, std::shared_ptr, unsigned long, std::map, std::allocator>>&, std::map, std::allocator>>&, bool, bool) @ 0x0000000012233991 in /usr/lib/debug/usr/bin/clickhouse.debug 14. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyzeSelect(std::shared_ptr&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector> const&, std::vector> const&, std::shared_ptr) const @ 0x000000001222f501 in /usr/lib/debug/usr/bin/clickhouse.debug 15. ./build_docker/./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr const&, std::shared_ptr const&, std::optional, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&, std::shared_ptr const&, std::shared_ptr)::$_0::operator()(bool) const @ 0x0000000011ee283c in /usr/lib/debug/usr/bin/clickhouse.debug 16. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr const&, std::shared_ptr const&, std::optional, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000011ed6895 in /usr/lib/debug/usr/bin/clickhouse.debug 17. ./build_docker/./contrib/llvm-project/libcxx/include/optional:260: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr const&, std::shared_ptr, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000011f86628 in /usr/lib/debug/usr/bin/clickhouse.debug 18. ./build_docker/./contrib/llvm-project/libcxx/include/vector:434: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000011e8c83e in /usr/lib/debug/usr/bin/clickhouse.debug 19. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122d12aa in /usr/lib/debug/usr/bin/clickhouse.debug 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122cd295 in /usr/lib/debug/usr/bin/clickhouse.debug 21. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000131543b9 in /usr/lib/debug/usr/bin/clickhouse.debug 22. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 24. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 26. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 27. ? @ 0x00007ffaf603cac3 in ? 28. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:00.418460 [ 725 ] {877e8b14-111a-4a54-9df2-4f246262ed4c} TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery4: While processing CounterID > ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery4). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135c8f66 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/vector:1420: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000013589dac in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x00000000135a1b2a in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x0000000011f8b6d9 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:188: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x00000000120118d7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::InDepthNodeVisitor>::doVisit(std::shared_ptr&) @ 0x00000000120101fd in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010359 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ./build_docker/./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000001201504a in /usr/lib/debug/usr/bin/clickhouse.debug 11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor>::doVisit(std::shared_ptr&) @ 0x000000001201021e in /usr/lib/debug/usr/bin/clickhouse.debug 12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor>::visitChildren(std::shared_ptr&) @ 0x0000000012010359 in /usr/lib/debug/usr/bin/clickhouse.debug 13. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1551: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr&, std::shared_ptr, unsigned long, std::map, std::allocator>>&, std::map, std::allocator>>&, bool, bool) @ 0x0000000012233991 in /usr/lib/debug/usr/bin/clickhouse.debug 14. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyzeSelect(std::shared_ptr&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector> const&, std::vector> const&, std::shared_ptr) const @ 0x000000001222f501 in /usr/lib/debug/usr/bin/clickhouse.debug 15. ./build_docker/./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr const&, std::shared_ptr const&, std::optional, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&, std::shared_ptr const&, std::shared_ptr)::$_0::operator()(bool) const @ 0x0000000011ee283c in /usr/lib/debug/usr/bin/clickhouse.debug 16. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr const&, std::shared_ptr const&, std::optional, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000011ed6895 in /usr/lib/debug/usr/bin/clickhouse.debug 17. ./build_docker/./contrib/llvm-project/libcxx/include/optional:260: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr const&, std::shared_ptr, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000011f86628 in /usr/lib/debug/usr/bin/clickhouse.debug 18. ./build_docker/./contrib/llvm-project/libcxx/include/vector:434: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000011e8c83e in /usr/lib/debug/usr/bin/clickhouse.debug 19. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122d12aa in /usr/lib/debug/usr/bin/clickhouse.debug 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122cd295 in /usr/lib/debug/usr/bin/clickhouse.debug 21. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x00000000131543b9 in /usr/lib/debug/usr/bin/clickhouse.debug 22. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 24. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 26. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 27. ? @ 0x00007ffaf603cac3 in ? 28. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:00.438557 [ 725 ] {b9d99c6a-7d38-4c30-8d75-1281a2db6a8d} executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeThread. (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:51090) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:00.439825 [ 725 ] {b9d99c6a-7d38-4c30-8d75-1281a2db6a8d} TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeThread. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, String&&, String&&) @ 0x000000001132a8f6 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a77b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:45.567505 [ 1776 ] {3aa14d39-f05d-4858-8505-3e2a569f80ce} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 583.69 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:41280) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000001132aac0 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a5ed in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:45.570399 [ 1776 ] {3aa14d39-f05d-4858-8505-3e2a569f80ce} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 583.69 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000001132aac0 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a5ed in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:45.635099 [ 1776 ] {d9d36104-641a-45f1-aa4f-7532b543476a} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 576.99 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:41280) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000001132aac0 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a5ed in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:45.637905 [ 1776 ] {d9d36104-641a-45f1-aa4f-7532b543476a} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 576.99 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000001132aac0 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a5ed in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:45.704387 [ 1776 ] {91974bfe-a9c0-41a8-9f07-8499be08beb9} executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 585.24 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:41280) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000001132aac0 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a5ed in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:45.708031 [ 1776 ] {91974bfe-a9c0-41a8-9f07-8499be08beb9} TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 585.24 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000001132aac0 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000001132a5ed in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list> const&) @ 0x0000000011316a75 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x00000000131ba50f in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000131b12b0 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x00000000131b23ef in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x000000000c70b2be in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./src/Common/ThreadPool.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c70ed7c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.883546 [ 1775 ] {710d0b1e-573e-482f-b8b4-40ba997d21f1} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00202759 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:56736) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000011317e62 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000011317d80 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000131b08f1 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000131b0690 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000131bde4f in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ? @ 0x00007ffaf603cac3 in ? 8. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.899053 [ 1698 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54392). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000c8a31a7 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000c8a2dd7 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/ReadBuffer.h:72: DB::TCPHandler::runImpl() @ 0x0000000013153a4f in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:50.903451 [ 1771 ] {c1a656a7-d736-45a0-9b88-15fcc92b768b} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:55962). (NETWORK_ERROR) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55962) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001315f6e0 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000131560d2 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.903496 [ 725 ] {b8751fdf-d7af-470f-a5ac-6cb5cb9f4c5d} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:33818). (NETWORK_ERROR) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33818) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001315f6e0 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000131560d2 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.907469 [ 1770 ] {76990730-82e1-4fae-be01-22e2f73ca609} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:33274). (NETWORK_ERROR) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33274) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.907507 [ 1557 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:55544). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000c8a31a7 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000c8a2dd7 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/ReadBuffer.h:72: DB::TCPHandler::runImpl() @ 0x0000000013153a4f in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:50.907582 [ 1769 ] {cb5d4d17-984c-464d-9329-a3aa25edd9ff} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:60330). (NETWORK_ERROR) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60330) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.907591 [ 1775 ] {710d0b1e-573e-482f-b8b4-40ba997d21f1} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00202759 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000011317e62 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000011317d80 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000131b08f1 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000131b0690 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000131bde4f in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ? @ 0x00007ffaf603cac3 in ? 8. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.907573 [ 1776 ] {bfcf723e-d0fe-4ccc-997b-a3b777c9df70} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:33756). (NETWORK_ERROR) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33756) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.912999 [ 1770 ] {76990730-82e1-4fae-be01-22e2f73ca609} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:33274). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a485b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:50.913018 [ 1769 ] {cb5d4d17-984c-464d-9329-a3aa25edd9ff} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:60330). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a485b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:50.913076 [ 1776 ] {bfcf723e-d0fe-4ccc-997b-a3b777c9df70} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:33756). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a485b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:50.913143 [ 1769 ] {cb5d4d17-984c-464d-9329-a3aa25edd9ff} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:60330). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.913115 [ 1771 ] {c1a656a7-d736-45a0-9b88-15fcc92b768b} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:55962). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a485b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:50.913132 [ 1770 ] {76990730-82e1-4fae-be01-22e2f73ca609} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:33274). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.913253 [ 1776 ] {bfcf723e-d0fe-4ccc-997b-a3b777c9df70} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:33756). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.913257 [ 1771 ] {c1a656a7-d736-45a0-9b88-15fcc92b768b} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:55962). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001315f6e0 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000131560d2 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.913378 [ 1769 ] {cb5d4d17-984c-464d-9329-a3aa25edd9ff} TCPHandler: Client has gone away. 2024.10.22 05:24:50.913473 [ 1770 ] {76990730-82e1-4fae-be01-22e2f73ca609} TCPHandler: Client has gone away. 2024.10.22 05:24:50.913581 [ 1771 ] {c1a656a7-d736-45a0-9b88-15fcc92b768b} TCPHandler: Client has gone away. 2024.10.22 05:24:50.913630 [ 1776 ] {bfcf723e-d0fe-4ccc-997b-a3b777c9df70} TCPHandler: Client has gone away. 2024.10.22 05:24:50.914292 [ 725 ] {b8751fdf-d7af-470f-a5ac-6cb5cb9f4c5d} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:33818). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a485b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:50.914326 [ 725 ] {b8751fdf-d7af-470f-a5ac-6cb5cb9f4c5d} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:33818). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000001315f6e0 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::TCPHandler::runImpl() @ 0x00000000131560d2 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:50.914442 [ 725 ] {b8751fdf-d7af-470f-a5ac-6cb5cb9f4c5d} TCPHandler: Client has gone away. 2024.10.22 05:24:53.048297 [ 1775 ] {91d33e08-4329-4d50-a0f4-a4d8ca38c1dc} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.012644907 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:56756) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000011317e62 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000011317d80 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000131b08f1 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000131b0690 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000131bde4f in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ? @ 0x00007ffaf603cac3 in ? 8. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.075163 [ 725 ] {8ce36aa3-0efa-44fd-b172-4c5ad8e19d0f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.005489372 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55576) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000011317e62 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000011317d80 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000131b08f1 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000131b0690 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000131bde4f in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ? @ 0x00007ffaf603cac3 in ? 8. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.086618 [ 1773 ] {d3d14b0d-1e9f-4831-8cc5-a09896ec0186} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008193685 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54402) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000011317e62 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000011317d80 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000131b08f1 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000131b0690 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000131bde4f in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ? @ 0x00007ffaf603cac3 in ? 8. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.089291 [ 1557 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:33294). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000c8a31a7 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000c8a2dd7 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/ReadBuffer.h:72: DB::TCPHandler::runImpl() @ 0x0000000013153a4f in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:53.089427 [ 1698 ] {49076f15-b716-432f-85ad-f8bab9fb4ddf} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:59098). (NETWORK_ERROR) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59098) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.090898 [ 1698 ] {49076f15-b716-432f-85ad-f8bab9fb4ddf} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:59098). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a485b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:53.090994 [ 1698 ] {49076f15-b716-432f-85ad-f8bab9fb4ddf} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:59098). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.091261 [ 1769 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:55980). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000c8a31a7 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000c8a2dd7 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/ReadBuffer.h:72: DB::TCPHandler::runImpl() @ 0x0000000013153a4f in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:53.091277 [ 1698 ] {49076f15-b716-432f-85ad-f8bab9fb4ddf} TCPHandler: Client has gone away. 2024.10.22 05:24:53.099797 [ 1771 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:33840). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000c8a31a7 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000c8a2dd7 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/ReadBuffer.h:72: DB::TCPHandler::runImpl() @ 0x0000000013153a4f in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:53.099851 [ 1773 ] {d3d14b0d-1e9f-4831-8cc5-a09896ec0186} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:54402). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:53.099979 [ 1773 ] {d3d14b0d-1e9f-4831-8cc5-a09896ec0186} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008193685 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000011317e62 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000011317d80 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000131b08f1 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000131b0690 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000131bde4f in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ? @ 0x00007ffaf603cac3 in ? 8. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.100010 [ 725 ] {8ce36aa3-0efa-44fd-b172-4c5ad8e19d0f} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:55576). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:53.100140 [ 725 ] {8ce36aa3-0efa-44fd-b172-4c5ad8e19d0f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.005489372 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000011317e62 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000011317d80 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000131b08f1 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000131b0690 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000131bde4f in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ? @ 0x00007ffaf603cac3 in ? 8. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.099983 [ 1770 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:33762). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000c8a31a7 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000c8a2dd7 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/ReadBuffer.h:72: DB::TCPHandler::runImpl() @ 0x0000000013153a4f in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:53.100369 [ 725 ] {8ce36aa3-0efa-44fd-b172-4c5ad8e19d0f} TCPHandler: Client has gone away. 2024.10.22 05:24:53.100570 [ 1773 ] {d3d14b0d-1e9f-4831-8cc5-a09896ec0186} TCPHandler: Client has gone away. 2024.10.22 05:24:53.112792 [ 1776 ] {f59d679a-adcb-4831-ad4a-7f611ca6322c} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:39226). (NETWORK_ERROR) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39226) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.112808 [ 1775 ] {91d33e08-4329-4d50-a0f4-a4d8ca38c1dc} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.012644907 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x0000000011317e62 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:0: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000011317d80 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./contrib/llvm-project/libcxx/include/atomic:958: DB::PipelineExecutor::finalizeExecution() @ 0x00000000131b08f1 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000131b0690 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000131bde4f in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/base/../base/wide_integer_impl.h:809: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x000000000c70d5c4 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ? @ 0x00007ffaf603cac3 in ? 8. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.113242 [ 1776 ] {f59d679a-adcb-4831-ad4a-7f611ca6322c} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:39226). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a485b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:53.113327 [ 1776 ] {f59d679a-adcb-4831-ad4a-7f611ca6322c} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:39226). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.113609 [ 1776 ] {f59d679a-adcb-4831-ad4a-7f611ca6322c} TCPHandler: Client has gone away. 2024.10.22 05:24:53.123740 [ 1772 ] {5ddeae19-adce-402f-b82e-c1905fc06ade} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:60346). (NETWORK_ERROR) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60346) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.124173 [ 1772 ] {5ddeae19-adce-402f-b82e-c1905fc06ade} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:60346). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a485b in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogs() @ 0x000000001315e7a2 in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:680: DB::TCPHandler::runImpl() @ 0x00000000131586cc in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 10. ? @ 0x00007ffaf603cac3 in ? 11. ? @ 0x00007ffaf60cda04 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 05:24:53.124268 [ 1772 ] {5ddeae19-adce-402f-b82e-c1905fc06ade} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:60346). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c62c697 in /usr/lib/debug/usr/bin/clickhouse.debug 1. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000c8a4b50 in /usr/lib/debug/usr/bin/clickhouse.debug 2. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000c8a46b1 in /usr/lib/debug/usr/bin/clickhouse.debug 3. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::runImpl() @ 0x000000001315753b in /usr/lib/debug/usr/bin/clickhouse.debug 4. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000013166799 in /usr/lib/debug/usr/bin/clickhouse.debug 5. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000015b58674 in /usr/lib/debug/usr/bin/clickhouse.debug 6. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b59871 in /usr/lib/debug/usr/bin/clickhouse.debug 7. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x0000000015c904a7 in /usr/lib/debug/usr/bin/clickhouse.debug 8. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c8e77c in /usr/lib/debug/usr/bin/clickhouse.debug 9. ? @ 0x00007ffaf603cac3 in ? 10. ? @ 0x00007ffaf60cda04 in ? 2024.10.22 05:24:53.124425 [ 1772 ] {5ddeae19-adce-402f-b82e-c1905fc06ade} TCPHandler: Client has gone away. 2024.10.22 05:25:24.622447 [ 722 ] {} Application: Closed all listening sockets. Waiting for 4 outstanding connections. 2024.10.22 05:25:29.528991 [ 722 ] {} Application: Closed connections. But 4 remain. Tip: To increase wait time add to config: 60 2024.10.22 05:25:29.529053 [ 722 ] {} Application: Will shutdown forcefully.