2024.10.21 19:01:14.813019 [ 348 ] {} <Warning> Context: Server was built with sanitizer. It will work slowly.
2024.10.21 19:01:14.814569 [ 348 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.10.21 19:01:16.274601 [ 348 ] {} <Warning> Context: The setting 'merge_tree_metadata_cache' is enabled. But the feature of 'metadata cache in RocksDB' is experimental and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. See the corresponding report at https://github.com/ClickHouse/ClickHouse/issues/51182
2024.10.21 19:01:16.500972 [ 348 ] {} <Warning> KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper.
2024.10.21 19:01:16.501841 [ 348 ] {} <Warning> KeeperLogStore: Removing all changelogs
2024.10.21 19:01:16.502668 [ 348 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.10.21 19:01:16.508025 [ 348 ] {} <Warning> KeeperStateManager: No state was read
2024.10.21 19:01:16.508055 [ 348 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.10.21 19:01:16.508076 [ 348 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.10.21 19:01:16.558670 [ 574 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.10.21 19:01:16.583295 [ 348 ] {} <Warning> Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist
2024.10.21 19:01:16.583340 [ 348 ] {} <Warning> Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/
2024.10.21 19:01:16.664383 [ 348 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.10.21 19:01:55.608024 [ 348 ] {} <Warning> KeeperLogStore: Changelog is shut down
2024.10.21 19:01:56.404246 [ 745 ] {} <Warning> Context: Server was built with sanitizer. It will work slowly.
2024.10.21 19:01:56.405502 [ 745 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.10.21 19:01:57.862028 [ 745 ] {} <Warning> Context: The setting 'merge_tree_metadata_cache' is enabled. But the feature of 'metadata cache in RocksDB' is experimental and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. See the corresponding report at https://github.com/ClickHouse/ClickHouse/issues/51182
2024.10.21 19:01:58.124524 [ 745 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.10.21 19:01:58.131602 [ 745 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.10.21 19:01:58.131635 [ 745 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.10.21 19:01:58.182195 [ 962 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.10.21 19:01:58.617808 [ 745 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.10.21 19:03:29.352918 [ 748 ] {b4549aee-be5c-4480-9ea3-ed3667e42264} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery1: While processing 1 = ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery1). (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:46976) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:233: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035cba279 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cf5b69 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c495d2 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c9ad33 in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x00000000316e6039 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000003185f205 in /usr/bin/clickhouse
10. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000003185e4bd in /usr/bin/clickhouse
11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x000000003185d9fc in /usr/bin/clickhouse
12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd49 in /usr/bin/clickhouse
13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x0000000031867e5c in /usr/bin/clickhouse
14. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x000000003185d9fc in /usr/bin/clickhouse
15. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd49 in /usr/bin/clickhouse
16. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd54 in /usr/bin/clickhouse
17. ./build_docker/./src/Interpreters/TreeRewriter.cpp:465: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x0000000031ee58c1 in /usr/bin/clickhouse
18. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1225: DB::TreeRewriter::analyzeSelect(std::shared_ptr<DB::IAST>&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector<DB::TableWithColumnNamesAndTypes, std::allocator<DB::TableWithColumnNamesAndTypes>> const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::TableJoin>) const @ 0x0000000031ed991f in /usr/bin/clickhouse
19. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:608: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>)::$_0::operator()(bool) const @ 0x0000000031516aa0 in /usr/bin/clickhouse
20. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x000000003150a011 in /usr/bin/clickhouse
21. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:206: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000031502217 in /usr/bin/clickhouse
22. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:257: DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::shared_ptr<DB::IAST> const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000316e1ab8 in /usr/bin/clickhouse
23. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000316dd9ba in /usr/bin/clickhouse
24. ./build_docker/./contrib/llvm-project/libcxx/include/vector:580: std::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectWithUnionQuery, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&) @ 0x000000003144e834 in /usr/bin/clickhouse
25. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x000000003144ba8f in /usr/bin/clickhouse
26. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000320eda3b in /usr/bin/clickhouse
27. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000320e5aa1 in /usr/bin/clickhouse
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed6d3d in /usr/bin/clickhouse
29. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
31. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse

2024.10.21 19:03:29.353768 [ 748 ] {b4549aee-be5c-4480-9ea3-ed3667e42264} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery1: While processing 1 = ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery1). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:233: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035cba279 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cf5b69 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c495d2 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c9ad33 in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x00000000316e6039 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000003185f205 in /usr/bin/clickhouse
10. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000003185e4bd in /usr/bin/clickhouse
11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x000000003185d9fc in /usr/bin/clickhouse
12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd49 in /usr/bin/clickhouse
13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x0000000031867e5c in /usr/bin/clickhouse
14. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x000000003185d9fc in /usr/bin/clickhouse
15. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd49 in /usr/bin/clickhouse
16. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd54 in /usr/bin/clickhouse
17. ./build_docker/./src/Interpreters/TreeRewriter.cpp:465: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x0000000031ee58c1 in /usr/bin/clickhouse
18. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1225: DB::TreeRewriter::analyzeSelect(std::shared_ptr<DB::IAST>&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector<DB::TableWithColumnNamesAndTypes, std::allocator<DB::TableWithColumnNamesAndTypes>> const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::TableJoin>) const @ 0x0000000031ed991f in /usr/bin/clickhouse
19. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:608: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>)::$_0::operator()(bool) const @ 0x0000000031516aa0 in /usr/bin/clickhouse
20. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x000000003150a011 in /usr/bin/clickhouse
21. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:206: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000031502217 in /usr/bin/clickhouse
22. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:257: DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::shared_ptr<DB::IAST> const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000316e1ab8 in /usr/bin/clickhouse
23. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000316dd9ba in /usr/bin/clickhouse
24. ./build_docker/./contrib/llvm-project/libcxx/include/vector:580: std::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectWithUnionQuery, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&) @ 0x000000003144e834 in /usr/bin/clickhouse
25. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x000000003144ba8f in /usr/bin/clickhouse
26. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000320eda3b in /usr/bin/clickhouse
27. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000320e5aa1 in /usr/bin/clickhouse
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed6d3d in /usr/bin/clickhouse
29. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
31. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse

2024.10.21 19:03:29.397999 [ 748 ] {8413b70e-01b0-4eca-9f09-cbf6e7e1894e} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:46976) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:233: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035cba279 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cf5b69 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c495d2 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c9ad33 in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x00000000316e6039 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000320eecdd in /usr/bin/clickhouse
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000320e5aa1 in /usr/bin/clickhouse
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed6d3d in /usr/bin/clickhouse
12. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
14. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
16. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
17. ? @ 0x00007f0557f00ac3 in ?
18. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:03:29.398506 [ 748 ] {8413b70e-01b0-4eca-9f09-cbf6e7e1894e} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:233: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035cba279 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cf5b69 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c495d2 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c9ad33 in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x00000000316e6039 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000320eecdd in /usr/bin/clickhouse
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000320e5aa1 in /usr/bin/clickhouse
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed6d3d in /usr/bin/clickhouse
12. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
14. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
16. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
17. ? @ 0x00007f0557f00ac3 in ?
18. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:03:29.574706 [ 748 ] {48eb64d9-6bd8-465b-9ddb-c1dab4495626} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeThread. (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:46976) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:03:29.583038 [ 748 ] {48eb64d9-6bd8-465b-9ddb-c1dab4495626} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeThread. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:03:29.656951 [ 748 ] {6647c145-3b52-445b-80ec-eefd10e08b3f} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery4: While processing CounterID > ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery4). (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:46976) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:233: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035cba279 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cf5b69 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c495d2 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c9ad33 in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x00000000316e6039 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000003185f205 in /usr/bin/clickhouse
10. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000003185e4bd in /usr/bin/clickhouse
11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x000000003185d9fc in /usr/bin/clickhouse
12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd49 in /usr/bin/clickhouse
13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x0000000031867e5c in /usr/bin/clickhouse
14. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x000000003185d9fc in /usr/bin/clickhouse
15. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd49 in /usr/bin/clickhouse
16. ./build_docker/./src/Interpreters/TreeRewriter.cpp:465: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x0000000031ee58c1 in /usr/bin/clickhouse
17. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1225: DB::TreeRewriter::analyzeSelect(std::shared_ptr<DB::IAST>&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector<DB::TableWithColumnNamesAndTypes, std::allocator<DB::TableWithColumnNamesAndTypes>> const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::TableJoin>) const @ 0x0000000031ed991f in /usr/bin/clickhouse
18. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:608: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>)::$_0::operator()(bool) const @ 0x0000000031516aa0 in /usr/bin/clickhouse
19. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x000000003150a011 in /usr/bin/clickhouse
20. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:206: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000031502217 in /usr/bin/clickhouse
21. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:257: DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::shared_ptr<DB::IAST> const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000316e1ab8 in /usr/bin/clickhouse
22. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000316dd9ba in /usr/bin/clickhouse
23. ./build_docker/./contrib/llvm-project/libcxx/include/vector:580: std::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectWithUnionQuery, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&) @ 0x000000003144e834 in /usr/bin/clickhouse
24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x000000003144ba8f in /usr/bin/clickhouse
25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000320eda3b in /usr/bin/clickhouse
26. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000320e5aa1 in /usr/bin/clickhouse
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed6d3d in /usr/bin/clickhouse
28. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
30. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse

2024.10.21 19:03:29.657515 [ 748 ] {6647c145-3b52-445b-80ec-eefd10e08b3f} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery4: While processing CounterID > ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery4). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:233: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x0000000035cba279 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035cf5b69 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c495d2 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000035c9ad33 in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x00000000316e6039 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000003185f205 in /usr/bin/clickhouse
10. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x000000003185e4bd in /usr/bin/clickhouse
11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x000000003185d9fc in /usr/bin/clickhouse
12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd49 in /usr/bin/clickhouse
13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x0000000031867e5c in /usr/bin/clickhouse
14. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x000000003185d9fc in /usr/bin/clickhouse
15. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x000000003185dd49 in /usr/bin/clickhouse
16. ./build_docker/./src/Interpreters/TreeRewriter.cpp:465: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x0000000031ee58c1 in /usr/bin/clickhouse
17. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1225: DB::TreeRewriter::analyzeSelect(std::shared_ptr<DB::IAST>&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector<DB::TableWithColumnNamesAndTypes, std::allocator<DB::TableWithColumnNamesAndTypes>> const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::TableJoin>) const @ 0x0000000031ed991f in /usr/bin/clickhouse
18. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:608: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>)::$_0::operator()(bool) const @ 0x0000000031516aa0 in /usr/bin/clickhouse
19. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x000000003150a011 in /usr/bin/clickhouse
20. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:206: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000031502217 in /usr/bin/clickhouse
21. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:257: DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::shared_ptr<DB::IAST> const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000316e1ab8 in /usr/bin/clickhouse
22. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000316dd9ba in /usr/bin/clickhouse
23. ./build_docker/./contrib/llvm-project/libcxx/include/vector:580: std::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectWithUnionQuery, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&) @ 0x000000003144e834 in /usr/bin/clickhouse
24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x000000003144ba8f in /usr/bin/clickhouse
25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000320eda3b in /usr/bin/clickhouse
26. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000320e5aa1 in /usr/bin/clickhouse
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed6d3d in /usr/bin/clickhouse
28. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
30. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse

2024.10.21 19:03:29.750200 [ 748 ] {0d4e1ce4-a629-4287-9534-2067d10e12bf} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeThread. (TOO_MANY_ROWS) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:46976) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:03:29.756637 [ 748 ] {0d4e1ce4-a629-4287-9534-2067d10e12bf} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeThread. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x000000002edd88d9 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd8085 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:33.694768 [ 1964 ] {2013d544-0ef3-4d17-92dc-7fb4afa15db0} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 586.35 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:38976) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002edd8f1e in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd823a in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:33.866817 [ 1923 ] {7fe78d4f-541b-4131-8912-9ebbee538ee9} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000020876 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45812) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:33.866858 [ 1964 ] {2013d544-0ef3-4d17-92dc-7fb4afa15db0} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 586.35 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002edd8f1e in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd823a in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:33.866880 [ 749 ] {be252086-d0c4-4f74-a0bc-5d0adc3a97c2} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00002089 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:39002) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:34.023244 [ 1923 ] {7fe78d4f-541b-4131-8912-9ebbee538ee9} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:45812). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:34.023247 [ 749 ] {be252086-d0c4-4f74-a0bc-5d0adc3a97c2} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00002089 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:34.023684 [ 1923 ] {7fe78d4f-541b-4131-8912-9ebbee538ee9} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000020876 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:34.039766 [ 1923 ] {7fe78d4f-541b-4131-8912-9ebbee538ee9} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:36.518725 [ 1964 ] {4e566e85-9d44-4a97-9116-f3564b876117} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 601.22 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:38976) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002edd8f1e in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd823a in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:36.839812 [ 749 ] {3a20c94e-af6a-4a1f-bf76-19a2b3ddb492} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.012020897 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:37708) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:36.840186 [ 1924 ] {c6b13f9a-c0ce-487f-881f-d27f7dcb9f96} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000020624 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34312) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:36.880140 [ 1964 ] {4e566e85-9d44-4a97-9116-f3564b876117} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 601.22 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002edd8f1e in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd823a in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:36.880177 [ 1923 ] {f72519aa-8fdd-4c8f-a150-7e885ac83976} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.314535199 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44398) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:36.983287 [ 1924 ] {c6b13f9a-c0ce-487f-881f-d27f7dcb9f96} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:34312). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:36.983444 [ 1924 ] {c6b13f9a-c0ce-487f-881f-d27f7dcb9f96} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000020624 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:36.989636 [ 1924 ] {c6b13f9a-c0ce-487f-881f-d27f7dcb9f96} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:37.002863 [ 1923 ] {f72519aa-8fdd-4c8f-a150-7e885ac83976} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:44398). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:37.005075 [ 1923 ] {f72519aa-8fdd-4c8f-a150-7e885ac83976} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.314535199 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:37.021944 [ 749 ] {3a20c94e-af6a-4a1f-bf76-19a2b3ddb492} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.012020897 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:37.023638 [ 1923 ] {f72519aa-8fdd-4c8f-a150-7e885ac83976} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:38.964813 [ 1964 ] {f3925f85-d88e-47f3-9fb6-3b6360fad681} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.61 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:38976) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002edd8f1e in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd823a in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:39.278142 [ 1964 ] {f3925f85-d88e-47f3-9fb6-3b6360fad681} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 572.61 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x000000002edd8f1e in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000002edd823a in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed98574 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:39.296660 [ 749 ] {24ea5492-cf9d-415e-8dfc-3544e4d8c361} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044021129 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:37714) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:39.316426 [ 1924 ] {e2dd5f04-8efe-49a6-91ec-6cc5d4632ac0} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.780880435 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50186) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:39.324786 [ 2894 ] {534ee847-71a6-4c25-8c6d-4f8eeeccb582} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028020777 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44412) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:39.324951 [ 1923 ] {7266132f-a1ed-40ed-840f-dcd15bd40311} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.024020701 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34318) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:39.408616 [ 1924 ] {e2dd5f04-8efe-49a6-91ec-6cc5d4632ac0} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.780880435 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:39.408996 [ 1924 ] {e2dd5f04-8efe-49a6-91ec-6cc5d4632ac0} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:39.412126 [ 2894 ] {534ee847-71a6-4c25-8c6d-4f8eeeccb582} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028020777 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:39.412497 [ 2894 ] {534ee847-71a6-4c25-8c6d-4f8eeeccb582} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:39.424227 [ 1923 ] {7266132f-a1ed-40ed-840f-dcd15bd40311} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.024020701 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:39.424684 [ 1923 ] {7266132f-a1ed-40ed-840f-dcd15bd40311} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:39.426212 [ 749 ] {24ea5492-cf9d-415e-8dfc-3544e4d8c361} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044021129 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:40.942394 [ 1964 ] {fb65f711-9a26-4ca7-84e1-fda134654a5c} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044020813 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:37730) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.413950 [ 1923 ] {9dbe095c-477b-4e25-959e-a2d8531be06e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.036020642 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34332) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.447671 [ 2894 ] {001ccd81-a7ec-417c-9341-3908835a566e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044020794 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44416) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.466664 [ 1924 ] {2456632d-f5a4-42c2-8070-b146739ba9a1} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.080021519 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42988) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.489116 [ 2895 ] {ac106e43-d046-408e-b858-eef1c574f993} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.048020873 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50192) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.641751 [ 1964 ] {fb65f711-9a26-4ca7-84e1-fda134654a5c} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044020813 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.706264 [ 1924 ] {2456632d-f5a4-42c2-8070-b146739ba9a1} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.080021519 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.714262 [ 1924 ] {2456632d-f5a4-42c2-8070-b146739ba9a1} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:41.718577 [ 1923 ] {9dbe095c-477b-4e25-959e-a2d8531be06e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.036020642 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.726433 [ 1923 ] {9dbe095c-477b-4e25-959e-a2d8531be06e} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:41.745270 [ 2894 ] {001ccd81-a7ec-417c-9341-3908835a566e} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:44416). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:41.745508 [ 2894 ] {001ccd81-a7ec-417c-9341-3908835a566e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044020794 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.750613 [ 2894 ] {001ccd81-a7ec-417c-9341-3908835a566e} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:41.752586 [ 2895 ] {ac106e43-d046-408e-b858-eef1c574f993} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.048020873 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:41.755535 [ 2895 ] {ac106e43-d046-408e-b858-eef1c574f993} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:43.290612 [ 1964 ] {9c94e58d-1aef-426b-ac69-e1bdc3aa9210} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116021912 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:37762) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.084722 [ 1923 ] {9c5b788a-8af8-4ac5-8789-5ce6a2e32371} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.052020649 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42992) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.088796 [ 1924 ] {d203e78c-1453-4039-8674-77728aa1c41a} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.060020797 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50198) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.089802 [ 2894 ] {5fd82021-ba78-4ef0-b704-fe5a73bc3087} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.026461806 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34336) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.129140 [ 2895 ] {0881881b-96a6-4b9b-a7b4-78b557b6fb4a} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004019695 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44428) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.149413 [ 2896 ] {ff3678ec-91f5-4aa5-906e-9ec1e8d04e47} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.08782226 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57074) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.171881 [ 1964 ] {9c94e58d-1aef-426b-ac69-e1bdc3aa9210} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116021912 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.209950 [ 2894 ] {5fd82021-ba78-4ef0-b704-fe5a73bc3087} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.026461806 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.212890 [ 2894 ] {5fd82021-ba78-4ef0-b704-fe5a73bc3087} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:44.227347 [ 2895 ] {0881881b-96a6-4b9b-a7b4-78b557b6fb4a} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:44428). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:44.227498 [ 2895 ] {0881881b-96a6-4b9b-a7b4-78b557b6fb4a} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004019695 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.228101 [ 2895 ] {0881881b-96a6-4b9b-a7b4-78b557b6fb4a} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:44.228387 [ 1923 ] {9c5b788a-8af8-4ac5-8789-5ce6a2e32371} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.052020649 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.232442 [ 1923 ] {9c5b788a-8af8-4ac5-8789-5ce6a2e32371} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:44.256479 [ 1924 ] {d203e78c-1453-4039-8674-77728aa1c41a} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.060020797 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.257702 [ 1924 ] {d203e78c-1453-4039-8674-77728aa1c41a} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:44.276394 [ 2896 ] {ff3678ec-91f5-4aa5-906e-9ec1e8d04e47} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.08782226 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:44.277327 [ 2896 ] {ff3678ec-91f5-4aa5-906e-9ec1e8d04e47} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:47.223312 [ 1964 ] {bff88883-aa8a-4ae4-8040-219754b36738} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.292024951 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:53662) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.645820 [ 2896 ] {a249a722-84d0-4deb-8ac8-8d8c5597f287} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.092021082 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42770) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.654057 [ 1924 ] {f3cbdc7e-1dd4-4edf-84b1-f518ae83986d} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.188022933 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33254) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.657080 [ 1923 ] {9f0d2606-1631-41f7-aff6-97297c7a3964} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116021543 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40404) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.657621 [ 2897 ] {7398e149-6513-4d65-94cf-079f613d3547} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.164022466 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50694) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.659840 [ 2895 ] {a873d8dc-0ffc-4045-8e12-2100862b9cda} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088021001 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36980) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.701494 [ 2894 ] {4bd00a09-2f68-414b-aa4d-a723b79c1232} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.120021611 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40954) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.766728 [ 1964 ] {bff88883-aa8a-4ae4-8040-219754b36738} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.292024951 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.787860 [ 2895 ] {a873d8dc-0ffc-4045-8e12-2100862b9cda} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088021001 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.793014 [ 1924 ] {f3cbdc7e-1dd4-4edf-84b1-f518ae83986d} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:33254). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:47.793175 [ 1924 ] {f3cbdc7e-1dd4-4edf-84b1-f518ae83986d} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.188022933 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.796472 [ 2895 ] {a873d8dc-0ffc-4045-8e12-2100862b9cda} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:47.803194 [ 1924 ] {f3cbdc7e-1dd4-4edf-84b1-f518ae83986d} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:47.813888 [ 1923 ] {9f0d2606-1631-41f7-aff6-97297c7a3964} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:40404). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:47.814097 [ 1923 ] {9f0d2606-1631-41f7-aff6-97297c7a3964} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116021543 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.820414 [ 1923 ] {9f0d2606-1631-41f7-aff6-97297c7a3964} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:47.821716 [ 2897 ] {7398e149-6513-4d65-94cf-079f613d3547} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.164022466 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.823779 [ 2894 ] {4bd00a09-2f68-414b-aa4d-a723b79c1232} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.120021611 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.826317 [ 2897 ] {7398e149-6513-4d65-94cf-079f613d3547} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:47.827412 [ 2894 ] {4bd00a09-2f68-414b-aa4d-a723b79c1232} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:47.844349 [ 2896 ] {a249a722-84d0-4deb-8ac8-8d8c5597f287} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.092021082 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:47.846324 [ 2896 ] {a249a722-84d0-4deb-8ac8-8d8c5597f287} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:49.660962 [ 1964 ] {4dcb1e8c-a22b-45c3-815a-2ecfa2e32998} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.160021888 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:53664) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.565773 [ 2894 ] {58327c85-1948-4db7-ab5c-97ba407d85d1} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.316024793 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33258) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.652153 [ 2898 ] {83cf8de4-b0bd-4353-ad49-df000f2baf2c} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.12402119 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40964) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.655612 [ 2895 ] {1e21205e-c582-4f33-aba7-55521c5943b2} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.036019543 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36994) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.656938 [ 1923 ] {e1c0ac12-bd39-44ee-8745-b5ca4a198d8c} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00801902 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40418) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.657861 [ 1924 ] {c17b0d5f-b2d8-4a4b-b98f-9b9e9948c8c5} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.07202022 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42346) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.658293 [ 2896 ] {760efac6-9858-4294-974d-797c9b0d7626} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.344025322 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42784) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.697342 [ 2897 ] {a148c1a1-3f46-4883-a719-f8dffa1abf97} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.156021787 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50708) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.786672 [ 1964 ] {4dcb1e8c-a22b-45c3-815a-2ecfa2e32998} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.160021888 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.790403 [ 2894 ] {58327c85-1948-4db7-ab5c-97ba407d85d1} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.316024793 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.796988 [ 2894 ] {58327c85-1948-4db7-ab5c-97ba407d85d1} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:50.821853 [ 1923 ] {e1c0ac12-bd39-44ee-8745-b5ca4a198d8c} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.00801902 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.826984 [ 1923 ] {e1c0ac12-bd39-44ee-8745-b5ca4a198d8c} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:50.836654 [ 2898 ] {83cf8de4-b0bd-4353-ad49-df000f2baf2c} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.12402119 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.841496 [ 2898 ] {83cf8de4-b0bd-4353-ad49-df000f2baf2c} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:50.856599 [ 1924 ] {c17b0d5f-b2d8-4a4b-b98f-9b9e9948c8c5} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.07202022 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.860710 [ 2895 ] {1e21205e-c582-4f33-aba7-55521c5943b2} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.036019543 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.866650 [ 1924 ] {c17b0d5f-b2d8-4a4b-b98f-9b9e9948c8c5} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:50.866736 [ 2895 ] {1e21205e-c582-4f33-aba7-55521c5943b2} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:50.872951 [ 2896 ] {760efac6-9858-4294-974d-797c9b0d7626} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42784). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:50.873223 [ 2896 ] {760efac6-9858-4294-974d-797c9b0d7626} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.344025322 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.874817 [ 2896 ] {760efac6-9858-4294-974d-797c9b0d7626} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:50.915613 [ 2897 ] {a148c1a1-3f46-4883-a719-f8dffa1abf97} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.156021787 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:50.918663 [ 2897 ] {a148c1a1-3f46-4883-a719-f8dffa1abf97} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:52.594539 [ 1964 ] {c79f77d6-9a0c-4c85-a369-5765abffe966} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.040019252 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:53670) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.283228 [ 1923 ] {1c58a223-7139-437a-8ee5-c1373f6bb962} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.104479081 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37000) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.283256 [ 2896 ] {79bb1d16-7802-4a31-9e6d-4ebba8733664} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.032019089 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50722) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.285611 [ 2898 ] {e888beba-3c49-47e7-b186-cb2ee80162d3} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.102111549 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42794) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.285742 [ 2897 ] {aea28cf3-4692-4b24-a77b-f9bb54d33ec5} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.095591832 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36640) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.289284 [ 1924 ] {a36acdb9-9f16-4207-8d16-694194cf5b33} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.10765241 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42350) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.297236 [ 1965 ] {c9e28346-cf13-44bf-9b04-175999a8d171} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.19202204 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40420) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.299122 [ 2895 ] {ca8f6a1f-18e3-4ee0-9d53-ae2a893da687} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.324024458 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33270) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.314247 [ 2894 ] {332ec546-53e7-49c6-844e-b1a758ec8f11} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.121348541 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40980) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.385685 [ 1924 ] {a36acdb9-9f16-4207-8d16-694194cf5b33} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.10765241 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.387963 [ 2898 ] {e888beba-3c49-47e7-b186-cb2ee80162d3} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.102111549 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.388190 [ 1924 ] {a36acdb9-9f16-4207-8d16-694194cf5b33} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:53.392359 [ 2898 ] {e888beba-3c49-47e7-b186-cb2ee80162d3} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:53.403487 [ 1964 ] {c79f77d6-9a0c-4c85-a369-5765abffe966} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.040019252 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.404115 [ 2896 ] {79bb1d16-7802-4a31-9e6d-4ebba8733664} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.032019089 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.405364 [ 1923 ] {1c58a223-7139-437a-8ee5-c1373f6bb962} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.104479081 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.406333 [ 2896 ] {79bb1d16-7802-4a31-9e6d-4ebba8733664} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:53.407708 [ 1923 ] {1c58a223-7139-437a-8ee5-c1373f6bb962} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:53.422300 [ 2895 ] {ca8f6a1f-18e3-4ee0-9d53-ae2a893da687} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.324024458 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.424263 [ 2895 ] {ca8f6a1f-18e3-4ee0-9d53-ae2a893da687} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:53.426436 [ 2894 ] {332ec546-53e7-49c6-844e-b1a758ec8f11} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.121348541 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.427281 [ 2894 ] {332ec546-53e7-49c6-844e-b1a758ec8f11} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:53.435737 [ 2897 ] {aea28cf3-4692-4b24-a77b-f9bb54d33ec5} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.095591832 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x0000000034ffd7b4 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000034ffcea0 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:96: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000035027307 in /usr/bin/clickhouse
7. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
9. ? @ 0x00007f0557f00ac3 in ?
10. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.436233 [ 2897 ] {aea28cf3-4692-4b24-a77b-f9bb54d33ec5} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:53.448099 [ 1965 ] {c9e28346-cf13-44bf-9b04-175999a8d171} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.19202204 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:53.449015 [ 1965 ] {c9e28346-cf13-44bf-9b04-175999a8d171} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.018774 [ 1964 ] {ed731739-debd-42d6-b56c-755c374ed60b} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.160021111 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::1]:53674) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.040710 [ 2894 ] {4cc08751-5542-4374-ba48-c9837911c728} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.288023383 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36904) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.088955 [ 1924 ] {75910241-4760-4ba4-a92f-ec4e02941428} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.276023165 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40422) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.100732 [ 1965 ] {cacaf22a-4c2c-48ee-8aa0-acbece15386b} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.040018909 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33274) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.106421 [ 2897 ] {96f1172c-9af8-4372-95ea-2bf1ca8882b6} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.256022798 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40988) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.107110 [ 2895 ] {2192fecb-3526-4a88-bcfb-a33063f0cfab} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.284023311 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42802) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.110153 [ 748 ] {6e356fcb-9833-4ee3-9ff1-f30c8005d40e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.284023311 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36656) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.112316 [ 1923 ] {7b271a01-3c10-4ca1-8e85-421afa07e1be} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.236022437 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42362) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.157154 [ 2896 ] {66c03540-1dc0-4a5c-bc10-2d113cd8b128} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.276023159 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37012) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.163376 [ 2898 ] {5abf1759-aa9b-4a10-b17e-e8b256c630d8} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000018178 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.8.16.42.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50724) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.221432 [ 748 ] {6e356fcb-9833-4ee3-9ff1-f30c8005d40e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.284023311 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.222534 [ 748 ] {6e356fcb-9833-4ee3-9ff1-f30c8005d40e} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.308189 [ 2894 ] {4cc08751-5542-4374-ba48-c9837911c728} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.288023383 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.336290 [ 2894 ] {4cc08751-5542-4374-ba48-c9837911c728} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.369252 [ 1924 ] {75910241-4760-4ba4-a92f-ec4e02941428} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:40422). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:57.371763 [ 1924 ] {75910241-4760-4ba4-a92f-ec4e02941428} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.276023165 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.381612 [ 1924 ] {75910241-4760-4ba4-a92f-ec4e02941428} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.397776 [ 2895 ] {2192fecb-3526-4a88-bcfb-a33063f0cfab} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:42802). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001dd6c55b in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:96: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001dd6a58a in /usr/bin/clickhouse
4. ./build_docker/./src/IO/WriteBuffer.h:60: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x0000000034f12fcc in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x0000000034ef1d88 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ed8b98 in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f13b30 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcb69cf in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcb7797 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e196fbc in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e190068 in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.21 19:06:57.398844 [ 2895 ] {2192fecb-3526-4a88-bcfb-a33063f0cfab} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.284023311 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.408675 [ 2895 ] {2192fecb-3526-4a88-bcfb-a33063f0cfab} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.423359 [ 1923 ] {7b271a01-3c10-4ca1-8e85-421afa07e1be} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.236022437 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.426798 [ 2897 ] {96f1172c-9af8-4372-95ea-2bf1ca8882b6} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.256022798 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.432218 [ 1923 ] {7b271a01-3c10-4ca1-8e85-421afa07e1be} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.434466 [ 2898 ] {5abf1759-aa9b-4a10-b17e-e8b256c630d8} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000018178 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.436160 [ 2897 ] {96f1172c-9af8-4372-95ea-2bf1ca8882b6} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.437911 [ 2898 ] {5abf1759-aa9b-4a10-b17e-e8b256c630d8} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.441010 [ 2896 ] {66c03540-1dc0-4a5c-bc10-2d113cd8b128} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.276023159 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.445972 [ 1965 ] {cacaf22a-4c2c-48ee-8aa0-acbece15386b} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.040018909 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:06:57.450529 [ 2896 ] {66c03540-1dc0-4a5c-bc10-2d113cd8b128} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.452770 [ 1965 ] {cacaf22a-4c2c-48ee-8aa0-acbece15386b} <Warning> TCPHandler: Client has gone away.
2024.10.21 19:06:57.453938 [ 1964 ] {ed731739-debd-42d6-b56c-755c374ed60b} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.160021111 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e082673 in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5cce50 in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x000000002ed9b6fd in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x000000002ed9a7a3 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000002ed97df7 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003501c719 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:251: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000034ffee3c in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:359: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000035001cc2 in /usr/bin/clickhouse
8. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001d8b0c1c in /usr/bin/clickhouse
9. ./build_docker/./src/Common/ThreadPool.h:243: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001d8bb080 in /usr/bin/clickhouse
10. ./build_docker/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001d8a7712 in /usr/bin/clickhouse
11. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001d8b59ed in /usr/bin/clickhouse
12. ? @ 0x00007f0557f00ac3 in ?
13. ? @ 0x00007f0557f91a04 in ?

2024.10.21 19:09:28.404442 [ 745 ] {} <Warning> Application: Closed all listening sockets. Waiting for 3 outstanding connections.
2024.10.21 19:09:33.964729 [ 745 ] {} <Warning> KeeperLogStore: Changelog is shut down