2024.11.04 22:41:06.063059 [ 396 ] {} <Warning> Context: Server was built with sanitizer. It will work slowly.
2024.11.04 22:41:06.102657 [ 396 ] {} <Warning> Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl.
2024.11.04 22:41:06.104575 [ 396 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.11.04 22:41:07.401730 [ 396 ] {} <Warning> CloudPlacementInfo: Placement info has not been loaded
2024.11.04 22:41:07.587864 [ 396 ] {} <Warning> KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper.
2024.11.04 22:41:07.589314 [ 396 ] {} <Warning> KeeperLogStore: Removing all changelogs
2024.11.04 22:41:07.590680 [ 396 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.11.04 22:41:07.597289 [ 396 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.11.04 22:41:07.597338 [ 396 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.11.04 22:41:07.649019 [ 987 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.11.04 22:41:07.703462 [ 396 ] {} <Warning> Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist
2024.11.04 22:41:07.703564 [ 396 ] {} <Warning> Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/
2024.11.04 22:41:07.989973 [ 396 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.11.04 22:41:09.572601 [ 400 ] {2fb83f87-0cbb-4fa2-91c7-549bbc896613} <Warning> InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 <table definition>; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/' <table definition>; to create new table and attach data.
2024.11.04 22:41:17.978379 [ 400 ] {cf88cab2-f74d-45ec-bd01-a0aa775f2223} <Warning> InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1 <table definition>; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/' <table definition>; to create new table and attach data.
2024.11.04 22:49:30.314824 [ 1435 ] {98b14a82-3ea6-4c97-9102-c24623204c04} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58512) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000016233c9d
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c8fcf5
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c36342
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c62c3e
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000178054e9
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000017805190
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2111: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000017375b23
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000173509d1
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001734f3f4
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000017368d48
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000017350556
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001734f3f4
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001735a662
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001734630b
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x00000000173430ef
19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000001733e55a
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000017804810
21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000178011e0
22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000017805b10
23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x000000001777156b
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000017d5f91f
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000017d5b6d8
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000001957ee72
27. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010

2024.11.04 22:49:30.316818 [ 1435 ] {98b14a82-3ea6-4c97-9102-c24623204c04} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000016233c9d
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c8fcf5
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c36342
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c62c3e
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000178054e9
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000017805190
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2111: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000017375b23
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000173509d1
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001734f3f4
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000017368d48
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000017350556
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001734f3f4
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001735a662
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001734630b
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x00000000173430ef
19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000001733e55a
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000017804810
21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000178011e0
22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000017805b10
23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x000000001777156b
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000017d5f91f
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000017d5b6d8
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000001957ee72
27. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010

2024.11.04 22:49:30.481664 [ 1435 ] {4a6c7030-5cb8-42a5-b9f6-2e342a4258c6} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58512) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000016233c9d
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c8fcf5
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c36342
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c62c3e
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000178054e9
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000017805190
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000017d60397
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000017d5b6d8
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000001957ee72
13. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
19. __tsan_thread_start_func @ 0x000000000701e48f
20. ? @ 0x00007f57b2eedac3
21. ? @ 0x00007f57b2f7f850

2024.11.04 22:49:30.483761 [ 1435 ] {4a6c7030-5cb8-42a5-b9f6-2e342a4258c6} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000016233c9d
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c8fcf5
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c36342
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c62c3e
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000178054e9
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000017805190
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000017d60397
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000017d5b6d8
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000001957ee72
13. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
19. __tsan_thread_start_func @ 0x000000000701e48f
20. ? @ 0x00007f57b2eedac3
21. ? @ 0x00007f57b2f7f850

2024.11.04 22:49:30.863299 [ 1435 ] {96f7ea64-ba5e-47fa-8e25-9340528dc1f6} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58512) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 22:49:30.884958 [ 1435 ] {96f7ea64-ba5e-47fa-8e25-9340528dc1f6} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 22:49:31.062838 [ 1435 ] {44c837a4-4081-4137-83e5-789517019f1d} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58512) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000016233c9d
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c8fcf5
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c36342
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c62c3e
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000178054e9
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000017805190
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2111: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000017375b23
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000173509d1
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001734f3f4
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000017368d48
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000017350556
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001734658c
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x00000000173430ef
17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000001733e55a
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000017804810
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000178011e0
20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000017805b10
21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x000000001777156b
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000017d5f91f
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000017d5b6d8
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000001957ee72
25. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
30. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
31. __tsan_thread_start_func @ 0x000000000701e48f

2024.11.04 22:49:31.064334 [ 1435 ] {44c837a4-4081-4137-83e5-789517019f1d} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x0000000016233c9d
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c8fcf5
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c36342
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x0000000019c62c3e
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x00000000178054e9
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000017805190
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2111: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000017375b23
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000173509d1
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6642: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000001734f3f4
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000017368d48
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x0000000017350556
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000001734658c
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x00000000173430ef
17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000001733e55a
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000017804810
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x00000000178011e0
20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000017805b10
21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x000000001777156b
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000017d5f91f
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000017d5b6d8
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000001957ee72
25. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
30. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
31. __tsan_thread_start_func @ 0x000000000701e48f

2024.11.04 22:49:31.467533 [ 1435 ] {fb1c5e97-8dd6-4dd1-80d2-73f0815fb10b} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58512) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 22:49:31.518635 [ 1435 ] {fb1c5e97-8dd6-4dd1-80d2-73f0815fb10b} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000016233e13
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:0: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x00000000162339c3
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:03:47.592493 [ 1855 ] {0f86b12a-6e2c-4391-bb51-6933ae79d131} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 592.10 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38586) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000016234040
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000016233af0
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:03:47.694715 [ 1855 ] {0f86b12a-6e2c-4391-bb51-6933ae79d131} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 592.10 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000016234040
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000016233af0
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:03:55.781252 [ 1855 ] {f792da0b-b48d-4a09-ac8a-1caa054ab1bb} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 573.85 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38586) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000016234040
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000016233af0
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:03:56.180573 [ 1855 ] {f792da0b-b48d-4a09-ac8a-1caa054ab1bb} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 573.85 MiB: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000016234040
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000016233af0
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:02.218585 [ 1855 ] {6fb12aeb-362a-401e-87da-e8ee5b9b7532} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 590.92 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38586) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000016234040
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000016233af0
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:02.331567 [ 1855 ] {6fb12aeb-362a-401e-87da-e8ee5b9b7532} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 590.92 MiB: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000016234040
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000016233af0
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x0000000016214d43
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:05.310528 [ 3479 ] {a1667063-cf45-46cd-ac51-038e3ab89245} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.023725094 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38384) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:05.310624 [ 1854 ] {e1084496-0b03-424e-a535-47e57cda63b1} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.022251159 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56020) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:05.358218 [ 3479 ] {a1667063-cf45-46cd-ac51-038e3ab89245} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.023725094 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:05.361303 [ 3060 ] {c4c4730a-7629-4275-a47c-0a8e51a8e018} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.170277092 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32784) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:05.410181 [ 3060 ] {c4c4730a-7629-4275-a47c-0a8e51a8e018} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.170277092 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:05.410232 [ 1854 ] {e1084496-0b03-424e-a535-47e57cda63b1} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.022251159 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:05.410181 [ 3479 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:38384). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000000f13bbbc
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000f13b071
4. DB::ReadBuffer::next() @ 0x000000000759da9c
5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x000000001957e2ca
6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
12. __tsan_thread_start_func @ 0x000000000701e48f
13. ? @ 0x00007f57b2eedac3
14. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:05.411071 [ 1854 ] {e1084496-0b03-424e-a535-47e57cda63b1} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:07.000074 [ 3479 ] {2b192e90-a693-4a56-98cf-5107cb16d10e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.136524212 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56846) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:07.000135 [ 3481 ] {324c2666-415d-4ca7-9e48-6a80b46f8a28} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064016105 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38390) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:07.001495 [ 1854 ] {783903bf-a65a-485a-aa3d-1c388de0a710} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016015379 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56034) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:07.081194 [ 3481 ] {324c2666-415d-4ca7-9e48-6a80b46f8a28} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064016105 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:07.082783 [ 1854 ] {783903bf-a65a-485a-aa3d-1c388de0a710} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016015379 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:07.096530 [ 3060 ] {f23a1512-ad4d-4062-bae7-30718379e527} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.307983815 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32786) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:07.113499 [ 3479 ] {2b192e90-a693-4a56-98cf-5107cb16d10e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.136524212 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:07.117512 [ 3479 ] {2b192e90-a693-4a56-98cf-5107cb16d10e} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:07.119391 [ 3060 ] {f23a1512-ad4d-4062-bae7-30718379e527} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.307983815 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:07.120713 [ 1854 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:56034). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000000f13bbbc
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000f13b071
4. DB::ReadBuffer::next() @ 0x000000000759da9c
5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x000000001957e2ca
6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
12. __tsan_thread_start_func @ 0x000000000701e48f
13. ? @ 0x00007f57b2eedac3
14. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:08.677625 [ 3060 ] {22a6058c-701c-405d-80bf-714fd97b275b} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116244344 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32796) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:08.955675 [ 3060 ] {22a6058c-701c-405d-80bf-714fd97b275b} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.116244344 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:09.123464 [ 1854 ] {43bd83c9-fdc0-4aef-aad8-f07263bdc7c0} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.396021057 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56852) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:09.167115 [ 3479 ] {e4c11269-5306-43e7-9b20-eacc8b07ee70} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.308019725 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38398) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:09.184609 [ 3481 ] {e86e0ac1-f57c-49f8-83a4-c0793b3e11d3} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.07201617 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56046) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:09.218879 [ 3493 ] {31b21fe9-67db-487e-aec9-cdc716a36d5e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.2800193 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58714) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:10.532373 [ 3479 ] {e4c11269-5306-43e7-9b20-eacc8b07ee70} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:38398). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000f13e24d
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000f13d533
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001959d359
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendLogs() @ 0x000000001958de3a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000001957feca
7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:10.532406 [ 3481 ] {e86e0ac1-f57c-49f8-83a4-c0793b3e11d3} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56046). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000f13e24d
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000f13d533
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001959d359
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendLogs() @ 0x000000001958de3a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000001957feca
7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:10.532571 [ 3493 ] {31b21fe9-67db-487e-aec9-cdc716a36d5e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.2800193 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:10.533118 [ 3481 ] {e86e0ac1-f57c-49f8-83a4-c0793b3e11d3} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.07201617 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:10.535178 [ 3493 ] {31b21fe9-67db-487e-aec9-cdc716a36d5e} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:10.535511 [ 3479 ] {e4c11269-5306-43e7-9b20-eacc8b07ee70} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.308019725 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:10.535702 [ 3481 ] {e86e0ac1-f57c-49f8-83a4-c0793b3e11d3} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:10.543904 [ 3479 ] {e4c11269-5306-43e7-9b20-eacc8b07ee70} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:10.674341 [ 3060 ] {d651a4fe-4d00-4cf2-a0ee-00c6d19ed841} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.271333086 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32798) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:10.740201 [ 3060 ] {d651a4fe-4d00-4cf2-a0ee-00c6d19ed841} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.271333086 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:10.969940 [ 3496 ] {f23a7bba-b5d0-4bf8-b810-88f771cbaa24} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064015994 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44056) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:10.970624 [ 3498 ] {ff50cb42-5569-48ba-9a38-92d7744e2e35} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044015694 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56856) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:10.986128 [ 3499 ] {987a41b2-c536-4307-8e8c-907325ce5ed3} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.024015393 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56062) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:11.004748 [ 1854 ] {43bd83c9-fdc0-4aef-aad8-f07263bdc7c0} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.396021057 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:11.007478 [ 1854 ] {43bd83c9-fdc0-4aef-aad8-f07263bdc7c0} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:11.063216 [ 3498 ] {ff50cb42-5569-48ba-9a38-92d7744e2e35} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.044015694 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:11.066043 [ 3498 ] {ff50cb42-5569-48ba-9a38-92d7744e2e35} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:11.066816 [ 3495 ] {a558cbb7-751c-482f-9107-aeab88d51fb0} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020015326 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38410) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:11.084236 [ 3496 ] {f23a7bba-b5d0-4bf8-b810-88f771cbaa24} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064015994 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:11.085772 [ 3496 ] {f23a7bba-b5d0-4bf8-b810-88f771cbaa24} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:11.102998 [ 3499 ] {987a41b2-c536-4307-8e8c-907325ce5ed3} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56062). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000f13e24d
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000f13d533
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001959d359
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendLogs() @ 0x000000001958de3a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000001957feca
7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:11.103775 [ 3499 ] {987a41b2-c536-4307-8e8c-907325ce5ed3} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.024015393 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:11.105024 [ 3499 ] {987a41b2-c536-4307-8e8c-907325ce5ed3} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:11.248178 [ 3495 ] {a558cbb7-751c-482f-9107-aeab88d51fb0} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020015326 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:11.249248 [ 3495 ] {a558cbb7-751c-482f-9107-aeab88d51fb0} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:11.278874 [ 3497 ] {d783e92a-fae0-4916-baa0-646c9680e0a3} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.232018506 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58724) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:11.758289 [ 3497 ] {d783e92a-fae0-4916-baa0-646c9680e0a3} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.232018506 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:11.783729 [ 3497 ] {d783e92a-fae0-4916-baa0-646c9680e0a3} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:12.264505 [ 3060 ] {bf9584f1-b4fb-416c-a965-05b7ef446f0d} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.123538377 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32800) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.383386 [ 3060 ] {bf9584f1-b4fb-416c-a965-05b7ef446f0d} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.123538377 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.653492 [ 3498 ] {4c6db541-c22f-4187-a50b-b76d497bc03a} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000014993 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58734) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.700388 [ 3481 ] {abc68df7-9c41-4142-9f69-ea7d93de7c5f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.380020683 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56862) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.712557 [ 3493 ] {ca600aa3-6be7-441f-ac2b-abc22ab5a36f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.380020683 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51770) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.752264 [ 3496 ] {63f1279d-2aa0-4c06-8401-fa1515f81448} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.304019541 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56066) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.786860 [ 3498 ] {4c6db541-c22f-4187-a50b-b76d497bc03a} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.000014993 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.788769 [ 3498 ] {4c6db541-c22f-4187-a50b-b76d497bc03a} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:12.797189 [ 3493 ] {ca600aa3-6be7-441f-ac2b-abc22ab5a36f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.380020683 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.800854 [ 3493 ] {ca600aa3-6be7-441f-ac2b-abc22ab5a36f} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:12.807640 [ 3481 ] {abc68df7-9c41-4142-9f69-ea7d93de7c5f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.380020683 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.812630 [ 3481 ] {abc68df7-9c41-4142-9f69-ea7d93de7c5f} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:12.818399 [ 3479 ] {61ab1569-eb0a-4097-a3d4-eb72f311597f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.33201996 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44068) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.825092 [ 1854 ] {5d2cb9a6-e4f7-4d38-8b3a-584804b8624e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.49602242 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38418) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.942189 [ 1854 ] {5d2cb9a6-e4f7-4d38-8b3a-584804b8624e} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:38418). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000f13e24d
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000f13d533
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001959d359
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendLogs() @ 0x000000001958de3a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000001957feca
7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:12.943639 [ 1854 ] {5d2cb9a6-e4f7-4d38-8b3a-584804b8624e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.49602242 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:12.972191 [ 1854 ] {5d2cb9a6-e4f7-4d38-8b3a-584804b8624e} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:12.996461 [ 3496 ] {63f1279d-2aa0-4c06-8401-fa1515f81448} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56066). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000f13e24d
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000f13d533
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001959d359
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendLogs() @ 0x000000001958de3a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000001957feca
7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:12.999982 [ 3496 ] {63f1279d-2aa0-4c06-8401-fa1515f81448} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.304019541 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:13.038468 [ 3496 ] {63f1279d-2aa0-4c06-8401-fa1515f81448} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:13.368337 [ 3479 ] {61ab1569-eb0a-4097-a3d4-eb72f311597f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.33201996 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:13.370831 [ 3479 ] {61ab1569-eb0a-4097-a3d4-eb72f311597f} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:13.926515 [ 3060 ] {a4c8bfc8-c516-48c0-829e-a60e30a3d8b1} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.14271595 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32816) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.164553 [ 3060 ] {a4c8bfc8-c516-48c0-829e-a60e30a3d8b1} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.14271595 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.498859 [ 3499 ] {1b188c10-cf4e-4035-a09a-79fd2b9ee48c} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.108016565 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56874) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.512173 [ 3498 ] {cd48c9aa-a064-4ed7-ad39-7795ec6d2a76} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.236018475 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51780) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.526759 [ 3495 ] {ac00ccdf-67db-4701-a37e-f059a1c6c457} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.068015968 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56068) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.536010 [ 3497 ] {994db017-3e5a-4701-9f69-6480af239c15} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064015907 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38420) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.554228 [ 3481 ] {94d35f27-6229-4d8d-8fd8-ea359f268948} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.140017036 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51764) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.578093 [ 3493 ] {6c5993aa-bf82-4b77-84f6-76d0be65b429} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.072016022 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58748) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.614317 [ 3499 ] {1b188c10-cf4e-4035-a09a-79fd2b9ee48c} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.108016565 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.616589 [ 3499 ] {1b188c10-cf4e-4035-a09a-79fd2b9ee48c} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:14.627456 [ 3498 ] {cd48c9aa-a064-4ed7-ad39-7795ec6d2a76} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.236018475 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.630943 [ 3498 ] {cd48c9aa-a064-4ed7-ad39-7795ec6d2a76} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:14.632440 [ 3495 ] {ac00ccdf-67db-4701-a37e-f059a1c6c457} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:56068). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000f13e24d
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000f13d533
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001959d359
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendLogs() @ 0x000000001958de3a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000001957feca
7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:14.632778 [ 3495 ] {ac00ccdf-67db-4701-a37e-f059a1c6c457} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.068015968 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.641202 [ 3495 ] {ac00ccdf-67db-4701-a37e-f059a1c6c457} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:14.644901 [ 3497 ] {994db017-3e5a-4701-9f69-6480af239c15} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064015907 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.654072 [ 3497 ] {994db017-3e5a-4701-9f69-6480af239c15} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:14.675713 [ 3502 ] {5814bae0-b907-4ec6-8e87-5d48293e3d35} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.040015539 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44074) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.923182 [ 3493 ] {6c5993aa-bf82-4b77-84f6-76d0be65b429} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:58748). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000f13e24d
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000f13d533
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001959d359
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendLogs() @ 0x000000001958de3a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000001957feca
7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:14.924045 [ 3493 ] {6c5993aa-bf82-4b77-84f6-76d0be65b429} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.072016022 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:14.961890 [ 3493 ] {6c5993aa-bf82-4b77-84f6-76d0be65b429} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:15.066294 [ 3481 ] {94d35f27-6229-4d8d-8fd8-ea359f268948} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.140017036 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:15.073712 [ 3481 ] {94d35f27-6229-4d8d-8fd8-ea359f268948} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:15.208063 [ 3502 ] {5814bae0-b907-4ec6-8e87-5d48293e3d35} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.040015539 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:15.228563 [ 3502 ] {5814bae0-b907-4ec6-8e87-5d48293e3d35} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:15.984323 [ 3060 ] {543031bf-8e5a-40fb-9c2e-2cf3153e068d} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.093605409 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46806) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.257341 [ 3060 ] {543031bf-8e5a-40fb-9c2e-2cf3153e068d} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.093605409 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.396208 [ 3479 ] {b83905e1-8f35-4fb0-ad0b-196c04131dff} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.092016274 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40360) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.411896 [ 3496 ] {f5e0a7fc-450e-4a98-a767-65c5b07cb25c} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.068015915 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59950) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.435989 [ 3495 ] {41afaecd-2caa-4e58-84ac-e5bbad6fd45b} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.120016691 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44106) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.437001 [ 1854 ] {8b9fc7e2-17a5-420f-b9ce-09785127f71d} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.084016155 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53454) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.445095 [ 3499 ] {2f332570-737e-4c32-934e-38db221496d1} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028015321 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58408) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.445108 [ 3498 ] {455097a7-1cd8-489a-8f8c-f4df97c3582f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.096016334 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55914) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.476070 [ 3504 ] {5604d5ef-8798-446d-b27c-2605e3fda0d8} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064015855 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59246) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.499475 [ 3505 ] {5bcbf435-ff9a-4bed-a5a2-d8be267d0ec2} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.5040224 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57746) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.567362 [ 1854 ] {8b9fc7e2-17a5-420f-b9ce-09785127f71d} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.084016155 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.574372 [ 1854 ] {8b9fc7e2-17a5-420f-b9ce-09785127f71d} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:16.575033 [ 3495 ] {41afaecd-2caa-4e58-84ac-e5bbad6fd45b} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.120016691 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.575618 [ 3479 ] {b83905e1-8f35-4fb0-ad0b-196c04131dff} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.092016274 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.576140 [ 3498 ] {455097a7-1cd8-489a-8f8c-f4df97c3582f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.096016334 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.576797 [ 3495 ] {41afaecd-2caa-4e58-84ac-e5bbad6fd45b} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:16.581220 [ 3498 ] {455097a7-1cd8-489a-8f8c-f4df97c3582f} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:16.582683 [ 3479 ] {b83905e1-8f35-4fb0-ad0b-196c04131dff} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:16.584423 [ 3499 ] {2f332570-737e-4c32-934e-38db221496d1} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:58408). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000f13e24d
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000f13d533
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001959d359
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendLogs() @ 0x000000001958de3a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000001957feca
7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:16.587800 [ 3499 ] {2f332570-737e-4c32-934e-38db221496d1} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028015321 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.588859 [ 3504 ] {5604d5ef-8798-446d-b27c-2605e3fda0d8} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.064015855 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.589752 [ 3499 ] {2f332570-737e-4c32-934e-38db221496d1} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:16.599361 [ 3504 ] {5604d5ef-8798-446d-b27c-2605e3fda0d8} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:16.604132 [ 3505 ] {5bcbf435-ff9a-4bed-a5a2-d8be267d0ec2} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.5040224 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.611583 [ 3505 ] {5bcbf435-ff9a-4bed-a5a2-d8be267d0ec2} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:16.639401 [ 3496 ] {f5e0a7fc-450e-4a98-a767-65c5b07cb25c} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.068015915 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:16.647494 [ 3496 ] {f5e0a7fc-450e-4a98-a767-65c5b07cb25c} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:18.456896 [ 3496 ] {06571850-416c-4713-88e5-c0f5b6eed62a} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.612023923 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57754) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.524790 [ 3481 ] {cf770cc7-5ed5-4564-ac75-8cf2d86cc18a} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.736025764 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55922) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.525340 [ 3498 ] {cd5fa906-ab43-4ed1-8866-2f874c614f4f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.032015325 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55000) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.536568 [ 3496 ] {06571850-416c-4713-88e5-c0f5b6eed62a} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.612023923 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.553474 [ 3493 ] {2bd7674f-6b9e-4989-a7c5-7a402903bff3} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.196017754 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59960) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.559449 [ 3495 ] {fb865bf3-5f02-49e5-a13a-98c0bf67eecb} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.780026416 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53466) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.569961 [ 3497 ] {c65ce4ff-4ea2-4f77-b125-556c3be341ef} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.216018051 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58424) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.582722 [ 3499 ] {2dac264d-77b7-476d-876c-05fef1938997} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.25601864 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40368) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.590222 [ 3479 ] {8fd8969e-2a6c-4764-b7e2-58dc695490fd} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.796026652 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59256) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.637649 [ 3481 ] {cf770cc7-5ed5-4564-ac75-8cf2d86cc18a} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.736025764 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.648657 [ 3495 ] {fb865bf3-5f02-49e5-a13a-98c0bf67eecb} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.780026416 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.654975 [ 3498 ] {cd5fa906-ab43-4ed1-8866-2f874c614f4f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.032015325 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.666507 [ 3479 ] {8fd8969e-2a6c-4764-b7e2-58dc695490fd} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.796026652 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.671384 [ 1854 ] {abb5d250-45ee-4eda-bf8c-94a8b44a6748} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.208017926 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44118) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.702545 [ 3493 ] {2bd7674f-6b9e-4989-a7c5-7a402903bff3} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.196017754 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.714748 [ 3497 ] {c65ce4ff-4ea2-4f77-b125-556c3be341ef} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.216018051 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.721934 [ 3499 ] {2dac264d-77b7-476d-876c-05fef1938997} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.25601864 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.772560 [ 1854 ] {abb5d250-45ee-4eda-bf8c-94a8b44a6748} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.208017926 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.807223 [ 3060 ] {ffc7ae70-0c4d-4332-a55b-2c627942a2a1} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.3400199 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46814) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.869206 [ 1854 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:44118). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000000f13bbbc
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000f13b071
4. DB::ReadBuffer::next() @ 0x000000000759da9c
5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x000000001957e2ca
6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
12. __tsan_thread_start_func @ 0x000000000701e48f
13. ? @ 0x00007f57b2eedac3
14. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:18.872657 [ 3060 ] {ffc7ae70-0c4d-4332-a55b-2c627942a2a1} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.3400199 seconds, maximum: 1: While executing Remote. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:18.873302 [ 3497 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:58424). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000000f13bbbc
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:0: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000f13b071
4. DB::ReadBuffer::next() @ 0x000000000759da9c
5. ./build_docker/./src/Server/TCPHandler.cpp:333: DB::TCPHandler::runImpl() @ 0x000000001957e2ca
6. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
12. __tsan_thread_start_func @ 0x000000000701e48f
13. ? @ 0x00007f57b2eedac3
14. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:20.598937 [ 3060 ] {c49c3f72-5910-4137-8c87-f31ae2d1c349} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.288616537 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46828) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:20.874520 [ 3060 ] {c49c3f72-5910-4137-8c87-f31ae2d1c349} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.288616537 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x000000001793ffe0
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:198: DB::PipelineExecutor::finalizeExecution() @ 0x000000001961b4c8
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001961b108
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001962f19c
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
10. __tsan_thread_start_func @ 0x000000000701e48f
11. ? @ 0x00007f57b2eedac3
12. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.206034 [ 1854 ] {71ff5b18-e7b9-4090-bddd-7b644894e8e2} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.036015313 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44124) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.206939 [ 3497 ] {29112cee-2c39-448d-aaaa-41761e6a7f15} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.084016022 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59962) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.210447 [ 3493 ] {03b4496a-b353-474e-a1d3-571f75fb8c7c} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.056015609 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40380) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.239862 [ 3502 ] {60550e68-e28f-48ac-97bb-022f3f871287} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028015195 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55012) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.266744 [ 3481 ] {5b300100-5dd5-4886-adc3-26f6268773cd} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.144016909 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57756) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.272807 [ 3479 ] {621ad987-e583-4d16-806c-03dc59b45203} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.332019681 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58430) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.273651 [ 3499 ] {5e1b30f5-fb9e-4dc9-aa95-3c71febd697d} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.30001921 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55928) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.277178 [ 3495 ] {c457b07f-4a15-48d6-a7b5-ef809202adde} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.328019624 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53476) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.277994 [ 3498 ] {0072fd6b-151e-460f-a551-f6889be6a5b8} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.300019207 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54168) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.282062 [ 3496 ] {c5035853-74cc-48e5-834a-a3f782470aad} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.232018204 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59264) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.751444 [ 3497 ] {29112cee-2c39-448d-aaaa-41761e6a7f15} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.084016022 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.753402 [ 1854 ] {71ff5b18-e7b9-4090-bddd-7b644894e8e2} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:44124). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000f13e24d
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000000f13d533
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000001959d359
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendLogs() @ 0x000000001958de3a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000001957feca
7. ./build_docker/./src/Server/TCPHandler.cpp:2341: DB::TCPHandler::run() @ 0x000000001959d4c8
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000001d0083a3
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000001d008c12
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:202: Poco::PooledThread::run() @ 0x000000001d209d47
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000001d208010
12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001d2064ca
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:21.753530 [ 3502 ] {60550e68-e28f-48ac-97bb-022f3f871287} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.028015195 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.755395 [ 1854 ] {71ff5b18-e7b9-4090-bddd-7b644894e8e2} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.036015313 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.766360 [ 3502 ] {60550e68-e28f-48ac-97bb-022f3f871287} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:21.770416 [ 1854 ] {71ff5b18-e7b9-4090-bddd-7b644894e8e2} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:21.774010 [ 3481 ] {5b300100-5dd5-4886-adc3-26f6268773cd} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.144016909 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.783134 [ 3481 ] {5b300100-5dd5-4886-adc3-26f6268773cd} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:21.789584 [ 3497 ] {29112cee-2c39-448d-aaaa-41761e6a7f15} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:21.814495 [ 3493 ] {03b4496a-b353-474e-a1d3-571f75fb8c7c} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.056015609 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.816220 [ 3493 ] {03b4496a-b353-474e-a1d3-571f75fb8c7c} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:21.869529 [ 3496 ] {c5035853-74cc-48e5-834a-a3f782470aad} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.232018204 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.877284 [ 3496 ] {c5035853-74cc-48e5-834a-a3f782470aad} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:21.878049 [ 3499 ] {5e1b30f5-fb9e-4dc9-aa95-3c71febd697d} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.30001921 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.885428 [ 3495 ] {c457b07f-4a15-48d6-a7b5-ef809202adde} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.328019624 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.887010 [ 3499 ] {5e1b30f5-fb9e-4dc9-aa95-3c71febd697d} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:21.902434 [ 3495 ] {c457b07f-4a15-48d6-a7b5-ef809202adde} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:21.949888 [ 3479 ] {621ad987-e583-4d16-806c-03dc59b45203} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.332019681 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.952221 [ 3479 ] {621ad987-e583-4d16-806c-03dc59b45203} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:04:21.962762 [ 3498 ] {0072fd6b-151e-460f-a551-f6889be6a5b8} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.300019207 seconds, maximum: 1: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001d17ad23
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000efa5034
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x0000000016216226
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000016215c4f
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x000000001621490c
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000001962ac11
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000001961cbb0
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000001961d231
8. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000f088372
9. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000000f08d6dd
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000000f08d642
11. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f083e00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f08a5f2
13. __tsan_thread_start_func @ 0x000000000701e48f
14. ? @ 0x00007f57b2eedac3
15. ? @ 0x00007f57b2f7f850

2024.11.04 23:04:21.966783 [ 3498 ] {0072fd6b-151e-460f-a551-f6889be6a5b8} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:19:21.109552 [ 396 ] {} <Warning> Application: Closed all listening sockets. Waiting for 4 outstanding connections.
2024.11.04 23:19:30.351940 [ 990 ] {} <Error> RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled