2024.03.19 19:09:05.332308 [ 339 ] {} <Warning> Context: Server was built with sanitizer. It will work slowly.
2024.03.19 19:09:05.332447 [ 339 ] {} <Warning> Context: Server logging level is set to 'test' and performance is degraded. This cannot be used in production.
2024.03.19 19:09:05.333656 [ 339 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.03.19 19:09:06.939864 [ 339 ] {} <Warning> KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper.
2024.03.19 19:09:06.940589 [ 339 ] {} <Warning> KeeperLogStore: Removing all changelogs
2024.03.19 19:09:06.941394 [ 339 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.03.19 19:09:06.941421 [ 339 ] {} <Fatal> KeeperServer: Can't start NON-SECURE keeper server in FIPS mode, please check the config.
2024.03.19 19:09:06.947212 [ 339 ] {} <Warning> KeeperStateManager: No state was read
2024.03.19 19:09:06.947278 [ 339 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.03.19 19:09:06.947303 [ 339 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.03.19 19:09:06.998248 [ 576 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.03.19 19:09:07.021037 [ 339 ] {} <Warning> Access(local directory): File /var/lib/clickhouse/access/users.list doesn't exist
2024.03.19 19:09:07.021076 [ 339 ] {} <Warning> Access(local directory): Recovering lists in directory /var/lib/clickhouse/access/
2024.03.19 19:09:07.071426 [ 339 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.03.19 19:09:37.547977 [ 339 ] {} <Warning> KeeperLogStore: Changelog is shut down
2024.03.19 19:09:39.354533 [ 729 ] {} <Warning> Context: Server was built with sanitizer. It will work slowly.
2024.03.19 19:09:39.354704 [ 729 ] {} <Warning> Context: Server logging level is set to 'test' and performance is degraded. This cannot be used in production.
2024.03.19 19:09:39.356334 [ 729 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.03.19 19:09:40.947789 [ 729 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.03.19 19:09:40.947821 [ 729 ] {} <Fatal> KeeperServer: Can't start NON-SECURE keeper server in FIPS mode, please check the config.
2024.03.19 19:09:40.951422 [ 729 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.03.19 19:09:40.951450 [ 729 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.03.19 19:09:41.002117 [ 948 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.03.19 19:09:41.121628 [ 1031 ] {} <Error> Container already exists, returning the existing container: std::exception. Code: 1001, type: Azure::Storage::StorageException, e.what() = 409 The specified container already exists.
The specified container already exists.
RequestId:df5a3206-2d36-4c76-ba70-dd1a8657ef4f
Time:2024-03-19T17:09:41.077Z
Request ID: df5a3206-2d36-4c76-ba70-dd1a8657ef4f, Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: std::runtime_error::runtime_error(String const&) @ 0x4642f22d in /usr/bin/clickhouse
1. ./build_docker/./contrib/azure/sdk/core/azure-core/inc/azure/core/exception.hpp:29: Azure::Storage::StorageException::CreateFromResponse(std::unique_ptr<Azure::Core::Http::RawResponse, std::default_delete<Azure::Core::Http::RawResponse>>) @ 0x383ff6cb in /usr/bin/clickhouse
2. ./build_docker/./contrib/azure/sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/protocol/blob_rest_client.hpp:4760: Azure::Storage::Blobs::_detail::BlobRestClient::BlobContainer::Create(Azure::Core::Http::_internal::HttpPipeline&, Azure::Core::Url const&, Azure::Storage::Blobs::_detail::BlobRestClient::BlobContainer::CreateBlobContainerOptions const&, Azure::Core::Context const&) @ 0x382af811 in /usr/bin/clickhouse
3. ./build_docker/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_container_client.cpp:0: Azure::Storage::Blobs::BlobContainerClient::Create(Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const @ 0x382a5cd1 in /usr/bin/clickhouse
4. ./build_docker/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_service_client.cpp:0: Azure::Storage::Blobs::BlobServiceClient::CreateBlobContainer(String const&, Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const @ 0x38310fd3 in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::getAzureBlobContainerClient(Poco::Util::AbstractConfiguration const&, String const&) @ 0x3118308e in /usr/bin/clickhouse
6. ./build_docker/./src/Disks/ObjectStorages/AzureBlobStorage/registerDiskAzureBlobStorage.cpp:0: std::shared_ptr<DB::IDisk> std::__function::__policy_invoker<std::shared_ptr<DB::IDisk> (String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerDiskAzureBlobStorage(DB::DiskFactory&, bool)::$_0, std::shared_ptr<DB::IDisk> (String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&)>>(std::__function::__policy_storage const*, String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>&&, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) @ 0x3117c127 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: DB::DiskFactory::create(String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) const @ 0x30fdfb02 in /usr/bin/clickhouse
8. ./build_docker/./src/Disks/DiskSelector.cpp:49: DB::DiskSelector::initialize(Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>) @ 0x3160e017 in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Context::getDiskSelector(std::lock_guard<std::mutex>&) const @ 0x31557d82 in /usr/bin/clickhouse
10. ./build_docker/./src/Interpreters/Context.cpp:3136: DB::Context::getStoragePolicySelector(std::lock_guard<std::mutex>&) const @ 0x31515751 in /usr/bin/clickhouse
11. ./build_docker/./src/Interpreters/Context.cpp:0: DB::Context::getStoragePolicy(String const&) const @ 0x31558bbc in /usr/bin/clickhouse
12. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::getStoragePolicy() const @ 0x35dddbfd in /usr/bin/clickhouse
13. ./build_docker/./src/Storages/MergeTree/MergeTreeData.h:0: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x35dd07c0 in /usr/bin/clickhouse
14. ./build_docker/./src/Storages/StorageMergeTree.cpp:112: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, bool, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool) @ 0x366a634d in /usr/bin/clickhouse
15. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool const&, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, bool const&) @ 0x366a4f8a in /usr/bin/clickhouse
16. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x3669a2fc in /usr/bin/clickhouse
17. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, bool) const @ 0x34ddbbc0 in /usr/bin/clickhouse
18. ./build_docker/./src/Databases/DatabaseOnDisk.cpp:110: DB::createTableFromAST(DB::ASTCreateQuery, String const&, String const&, std::shared_ptr<DB::Context>, bool) @ 0x30e838c3 in /usr/bin/clickhouse
19. ./build_docker/./src/Databases/DatabaseOrdinary.cpp:60: DB::DatabaseOrdinary::loadTableFromMetadata(std::shared_ptr<DB::Context>, String const&, DB::QualifiedTableName const&, std::shared_ptr<DB::IAST> const&, DB::LoadingStrictnessLevel) @ 0x30ed89da in /usr/bin/clickhouse
20. ./build_docker/./src/Databases/TablesLoader.cpp:208: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::TablesLoader::startLoadingTables(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, std::shared_ptr<DB::Context>, std::vector<DB::StorageID, std::allocator<DB::StorageID>> const&, unsigned long)::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x30fc07a2 in /usr/bin/clickhouse
21. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x2061fa0f in /usr/bin/clickhouse
22. ./build_docker/./src/Common/ThreadPool.h:228: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x2062a271 in /usr/bin/clickhouse
23. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
24. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
25. ? @ 0x7fc159366609 in ?
26. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:09:41.445884 [ 729 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.03.19 19:10:52.276084 [ 732 ] {24525cfa-814b-443a-a0f4-73cce8a21fb5} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery3: While processing 1 = ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery3). (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:46798) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:167: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x376b94f1 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x376ecdb5 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x37651558 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x3769d10c in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x33876548 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x3407805f in /usr/bin/clickhouse
10. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x340771b0 in /usr/bin/clickhouse
11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x33f892f7 in /usr/bin/clickhouse
12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89609 in /usr/bin/clickhouse
13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x340808a8 in /usr/bin/clickhouse
14. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x33f892f7 in /usr/bin/clickhouse
15. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89609 in /usr/bin/clickhouse
16. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89614 in /usr/bin/clickhouse
17. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1551: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x33f6f931 in /usr/bin/clickhouse
18. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1225: DB::TreeRewriter::analyzeSelect(std::shared_ptr<DB::IAST>&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector<DB::TableWithColumnNamesAndTypes, std::allocator<DB::TableWithColumnNamesAndTypes>> const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::TableJoin>, bool, std::unordered_map<String, String, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, String>>>, std::unordered_map<String, String, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, String>>>) const @ 0x33f6447f in /usr/bin/clickhouse
19. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:587: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>)::$_4::operator()(bool) const @ 0x336db0f9 in /usr/bin/clickhouse
20. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x336d15f0 in /usr/bin/clickhouse
21. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:202: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x336ca6c3 in /usr/bin/clickhouse
22. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:257: DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::shared_ptr<DB::IAST> const&, std::vector<String, std::allocator<String>> const&) @ 0x338723f6 in /usr/bin/clickhouse
23. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x3386dd9a in /usr/bin/clickhouse
24. ./build_docker/./contrib/llvm-project/libcxx/include/vector:580: std::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectWithUnionQuery, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&) @ 0x33629554 in /usr/bin/clickhouse
25. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x336267af in /usr/bin/clickhouse
26. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x34158dc5 in /usr/bin/clickhouse
27. ./build_docker/./src/Interpreters/executeQuery.cpp:1160: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x3415474c in /usr/bin/clickhouse
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae2c3e in /usr/bin/clickhouse
29. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
31. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse

2024.03.19 19:10:52.276793 [ 732 ] {24525cfa-814b-443a-a0f4-73cce8a21fb5} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery3: While processing 1 = ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery3). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:167: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x376b94f1 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x376ecdb5 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x37651558 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x3769d10c in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x33876548 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x3407805f in /usr/bin/clickhouse
10. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x340771b0 in /usr/bin/clickhouse
11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x33f892f7 in /usr/bin/clickhouse
12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89609 in /usr/bin/clickhouse
13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x340808a8 in /usr/bin/clickhouse
14. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x33f892f7 in /usr/bin/clickhouse
15. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89609 in /usr/bin/clickhouse
16. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89614 in /usr/bin/clickhouse
17. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1551: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x33f6f931 in /usr/bin/clickhouse
18. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1225: DB::TreeRewriter::analyzeSelect(std::shared_ptr<DB::IAST>&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector<DB::TableWithColumnNamesAndTypes, std::allocator<DB::TableWithColumnNamesAndTypes>> const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::TableJoin>, bool, std::unordered_map<String, String, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, String>>>, std::unordered_map<String, String, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, String>>>) const @ 0x33f6447f in /usr/bin/clickhouse
19. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:587: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>)::$_4::operator()(bool) const @ 0x336db0f9 in /usr/bin/clickhouse
20. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x336d15f0 in /usr/bin/clickhouse
21. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:202: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x336ca6c3 in /usr/bin/clickhouse
22. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:257: DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::shared_ptr<DB::IAST> const&, std::vector<String, std::allocator<String>> const&) @ 0x338723f6 in /usr/bin/clickhouse
23. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x3386dd9a in /usr/bin/clickhouse
24. ./build_docker/./contrib/llvm-project/libcxx/include/vector:580: std::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectWithUnionQuery, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&) @ 0x33629554 in /usr/bin/clickhouse
25. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x336267af in /usr/bin/clickhouse
26. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x34158dc5 in /usr/bin/clickhouse
27. ./build_docker/./src/Interpreters/executeQuery.cpp:1160: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x3415474c in /usr/bin/clickhouse
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae2c3e in /usr/bin/clickhouse
29. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
31. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse

2024.03.19 19:10:52.323630 [ 732 ] {2d951cd4-73d8-465e-8bf4-55d767d58cd0} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:46798) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:167: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x376b94f1 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x376ecdb5 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x37651558 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x3769d10c in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x33876548 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x3415b446 in /usr/bin/clickhouse
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1160: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x3415474c in /usr/bin/clickhouse
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae2c3e in /usr/bin/clickhouse
12. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
14. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
16. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
17. ? @ 0x7fc159366609 in ?
18. clone @ 0x7fc15928b353 in ?

2024.03.19 19:10:52.324447 [ 732 ] {2d951cd4-73d8-465e-8bf4-55d767d58cd0} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:167: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x376b94f1 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x376ecdb5 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x37651558 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x3769d10c in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x33876548 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x3415b446 in /usr/bin/clickhouse
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1160: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x3415474c in /usr/bin/clickhouse
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae2c3e in /usr/bin/clickhouse
12. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
14. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
16. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
17. ? @ 0x7fc159366609 in ?
18. clone @ 0x7fc15928b353 in ?

2024.03.19 19:10:52.390382 [ 732 ] {8bbb0ae9-f9c4-4e43-b182-75bd0e69092f} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeThread. (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:46798) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a817 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:10:52.392102 [ 732 ] {8bbb0ae9-f9c4-4e43-b182-75bd0e69092f} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeThread. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a817 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:10:52.461842 [ 732 ] {a35d48c0-cb6b-424d-899c-afa0fdb34a13} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery6: While processing CounterID > ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery6). (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:46798) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:167: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x376b94f1 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x376ecdb5 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x37651558 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x3769d10c in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x33876548 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x3407805f in /usr/bin/clickhouse
10. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x340771b0 in /usr/bin/clickhouse
11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x33f892f7 in /usr/bin/clickhouse
12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89609 in /usr/bin/clickhouse
13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x340808a8 in /usr/bin/clickhouse
14. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x33f892f7 in /usr/bin/clickhouse
15. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89609 in /usr/bin/clickhouse
16. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1551: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x33f6f931 in /usr/bin/clickhouse
17. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1225: DB::TreeRewriter::analyzeSelect(std::shared_ptr<DB::IAST>&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector<DB::TableWithColumnNamesAndTypes, std::allocator<DB::TableWithColumnNamesAndTypes>> const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::TableJoin>, bool, std::unordered_map<String, String, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, String>>>, std::unordered_map<String, String, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, String>>>) const @ 0x33f6447f in /usr/bin/clickhouse
18. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:587: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>)::$_4::operator()(bool) const @ 0x336db0f9 in /usr/bin/clickhouse
19. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x336d15f0 in /usr/bin/clickhouse
20. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:202: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x336ca6c3 in /usr/bin/clickhouse
21. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:257: DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::shared_ptr<DB::IAST> const&, std::vector<String, std::allocator<String>> const&) @ 0x338723f6 in /usr/bin/clickhouse
22. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x3386dd9a in /usr/bin/clickhouse
23. ./build_docker/./contrib/llvm-project/libcxx/include/vector:580: std::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectWithUnionQuery, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&) @ 0x33629554 in /usr/bin/clickhouse
24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x336267af in /usr/bin/clickhouse
25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x34158dc5 in /usr/bin/clickhouse
26. ./build_docker/./src/Interpreters/executeQuery.cpp:1160: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x3415474c in /usr/bin/clickhouse
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae2c3e in /usr/bin/clickhouse
28. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
30. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse

2024.03.19 19:10:52.462551 [ 732 ] {a35d48c0-cb6b-424d-899c-afa0fdb34a13} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery6: While processing CounterID > ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery6). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:167: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x376b94f1 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x376ecdb5 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x37651558 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x3769d10c in /usr/bin/clickhouse
8. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::execute() @ 0x33876548 in /usr/bin/clickhouse
9. ./build_docker/./src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x3407805f in /usr/bin/clickhouse
10. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x340771b0 in /usr/bin/clickhouse
11. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x33f892f7 in /usr/bin/clickhouse
12. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89609 in /usr/bin/clickhouse
13. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x340808a8 in /usr/bin/clickhouse
14. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x33f892f7 in /usr/bin/clickhouse
15. ./build_docker/./src/Interpreters/InDepthNodeVisitor.h:64: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x33f89609 in /usr/bin/clickhouse
16. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1551: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x33f6f931 in /usr/bin/clickhouse
17. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1225: DB::TreeRewriter::analyzeSelect(std::shared_ptr<DB::IAST>&, DB::TreeRewriterResult&&, DB::SelectQueryOptions const&, std::vector<DB::TableWithColumnNamesAndTypes, std::allocator<DB::TableWithColumnNamesAndTypes>> const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::TableJoin>, bool, std::unordered_map<String, String, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, String>>>, std::unordered_map<String, String, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, String>>>) const @ 0x33f6447f in /usr/bin/clickhouse
18. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:587: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>)::$_4::operator()(bool) const @ 0x336db0f9 in /usr/bin/clickhouse
19. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:0: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x336d15f0 in /usr/bin/clickhouse
20. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:202: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x336ca6c3 in /usr/bin/clickhouse
21. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:257: DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::shared_ptr<DB::IAST> const&, std::vector<String, std::allocator<String>> const&) @ 0x338723f6 in /usr/bin/clickhouse
22. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x3386dd9a in /usr/bin/clickhouse
23. ./build_docker/./contrib/llvm-project/libcxx/include/vector:580: std::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectWithUnionQuery, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&) @ 0x33629554 in /usr/bin/clickhouse
24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x336267af in /usr/bin/clickhouse
25. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x34158dc5 in /usr/bin/clickhouse
26. ./build_docker/./src/Interpreters/executeQuery.cpp:1160: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x3415474c in /usr/bin/clickhouse
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae2c3e in /usr/bin/clickhouse
28. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
30. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse

2024.03.19 19:10:52.644267 [ 732 ] {3ed7078a-bd10-4859-947f-335c8e9dae18} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeThread. (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:46798) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a817 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:10:52.645193 [ 732 ] {3ed7078a-bd10-4859-947f-335c8e9dae18} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeThread. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x309d3bc5 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d344f in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a817 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:37.532389 [ 3312 ] {e5a8a8f5-9eca-4916-bd47-a1b74e8c3b3e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.030293737 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38866) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:37.621687 [ 1305 ] {cbf77351-64be-44f1-bb9f-bf8c486921e3} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 578.05 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:53534) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x309d4155 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d35f5 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a817 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:37.656370 [ 3312 ] {e5a8a8f5-9eca-4916-bd47-a1b74e8c3b3e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.030293737 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:37.688698 [ 3311 ] {a3c21f75-e0f4-4f42-86ea-620178a847a8} <Error> executeQuery: Code: 159. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Timeout exceeded: elapsed 1.030293737 seconds, maximum: 1. Stack trace:

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?
: While executing Remote. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:37856) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x204b5a45 in /usr/bin/clickhouse
3. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x36847933 in /usr/bin/clickhouse
4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x36859b4b in /usr/bin/clickhouse
5. ./build_docker/./src/Client/PacketReceiver.h:0: DB::PacketReceiver::Routine::operator()(boost::context::fiber&&) @ 0x368a4dcf in /usr/bin/clickhouse
6. ./build_docker/./contrib/llvm-project/libcxx/include/__utility/swap.h:36: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, DB::PacketReceiver::Routine>::run() @ 0x368a4047 in /usr/bin/clickhouse
7. ./build_docker/./contrib/boost/boost/context/fiber_ucontext.hpp:74: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, DB::PacketReceiver::Routine>>(void*) @ 0x3689ee0b in /usr/bin/clickhouse

2024.03.19 19:13:37.688750 [ 1305 ] {cbf77351-64be-44f1-bb9f-bf8c486921e3} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 578.05 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x309d4155 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d35f5 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a817 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:37.756909 [ 3311 ] {a3c21f75-e0f4-4f42-86ea-620178a847a8} <Error> TCPHandler: Code: 159. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Timeout exceeded: elapsed 1.030293737 seconds, maximum: 1. Stack trace:

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?
: While executing Remote. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x204b5a45 in /usr/bin/clickhouse
3. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x36847933 in /usr/bin/clickhouse
4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x36859b4b in /usr/bin/clickhouse
5. ./build_docker/./src/Client/PacketReceiver.h:0: DB::PacketReceiver::Routine::operator()(boost::context::fiber&&) @ 0x368a4dcf in /usr/bin/clickhouse
6. ./build_docker/./contrib/llvm-project/libcxx/include/__utility/swap.h:36: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, DB::PacketReceiver::Routine>::run() @ 0x368a4047 in /usr/bin/clickhouse
7. ./build_docker/./contrib/boost/boost/context/fiber_ucontext.hpp:74: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, DB::PacketReceiver::Routine>>(void*) @ 0x3689ee0b in /usr/bin/clickhouse

2024.03.19 19:13:39.123715 [ 3311 ] {69350c32-8684-4513-8ca5-2d9405479bcf} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.102325506 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:37866) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:39.404513 [ 3313 ] {439f9a5f-071b-47d0-a8ef-00b2b38dcb97} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.283511968 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38882) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:39.412090 [ 3312 ] {b7bf1ae8-640b-446f-8fe5-ef05818577eb} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.284503312 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33860) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:39.504652 [ 3311 ] {69350c32-8684-4513-8ca5-2d9405479bcf} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.102325506 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:39.587388 [ 3313 ] {439f9a5f-071b-47d0-a8ef-00b2b38dcb97} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:38882). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x332c7985 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x332c63d1 in /usr/bin/clickhouse
4. ./build_docker/./src/IO/BufferBase.h:39: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x36b19b94 in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x36afd649 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae59aa in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
12. ? @ 0x7fc159366609 in ?
13. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:39.588611 [ 3313 ] {439f9a5f-071b-47d0-a8ef-00b2b38dcb97} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.283511968 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:39.596160 [ 3312 ] {b7bf1ae8-640b-446f-8fe5-ef05818577eb} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:33860). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x332c7985 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x332c63d1 in /usr/bin/clickhouse
4. ./build_docker/./src/IO/BufferBase.h:39: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x36b19b94 in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x36afd649 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae59aa in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
12. ? @ 0x7fc159366609 in ?
13. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:39.596419 [ 3312 ] {b7bf1ae8-640b-446f-8fe5-ef05818577eb} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.284503312 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:39.599904 [ 3313 ] {439f9a5f-071b-47d0-a8ef-00b2b38dcb97} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:39.607853 [ 3312 ] {b7bf1ae8-640b-446f-8fe5-ef05818577eb} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:39.637738 [ 1305 ] {4a30ee3d-1169-4175-b520-a9b6d7489da7} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 586.21 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:53534) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x309d4155 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d35f5 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a817 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:39.696071 [ 1305 ] {4a30ee3d-1169-4175-b520-a9b6d7489da7} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 586.21 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x309d4155 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x309d35f5 in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a817 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:41.300497 [ 3311 ] {68f8b997-7854-4614-bf3e-b1a6c58b1a0f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.527262436 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:37872) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:41.413901 [ 3314 ] {f2f59570-ecad-4ad5-b1f8-57e8ed68db5e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.575592439 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33864) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:41.415844 [ 3313 ] {e00def39-5774-4f33-aef5-311197b7dcaa} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.572271775 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48950) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:41.420223 [ 3312 ] {c1933d8f-f0e8-4f0c-b8b8-74c070635b54} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.580654139 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38894) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:41.520745 [ 3311 ] {68f8b997-7854-4614-bf3e-b1a6c58b1a0f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.527262436 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:41.540940 [ 3314 ] {f2f59570-ecad-4ad5-b1f8-57e8ed68db5e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.575592439 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:41.544302 [ 3314 ] {f2f59570-ecad-4ad5-b1f8-57e8ed68db5e} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:41.554335 [ 3312 ] {c1933d8f-f0e8-4f0c-b8b8-74c070635b54} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.580654139 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:41.558789 [ 3312 ] {c1933d8f-f0e8-4f0c-b8b8-74c070635b54} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:41.563782 [ 3313 ] {e00def39-5774-4f33-aef5-311197b7dcaa} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:48950). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x332c7985 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x332c63d1 in /usr/bin/clickhouse
4. ./build_docker/./src/IO/BufferBase.h:39: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x36b19b94 in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x36afd649 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae59aa in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
12. ? @ 0x7fc159366609 in ?
13. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:41.564091 [ 3313 ] {e00def39-5774-4f33-aef5-311197b7dcaa} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.572271775 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:41.567155 [ 3313 ] {e00def39-5774-4f33-aef5-311197b7dcaa} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:43.360554 [ 3312 ] {7759b65f-0bf5-49b4-aa1f-6ba32f08b0f7} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020025984 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38904) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:43.377854 [ 3316 ] {b9660834-b4be-4e63-8aa4-4bf32316802e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.511932294 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48964) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:43.388941 [ 3313 ] {7b2d051f-7196-4021-81ff-f40b9f17242c} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.534905807 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33876) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:43.394637 [ 3314 ] {299391d0-998b-460c-8e91-6adc35ccb974} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.531638486 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51682) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:43.468735 [ 3313 ] {7b2d051f-7196-4021-81ff-f40b9f17242c} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.534905807 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:43.474210 [ 3314 ] {299391d0-998b-460c-8e91-6adc35ccb974} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.531638486 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:43.478510 [ 3312 ] {7759b65f-0bf5-49b4-aa1f-6ba32f08b0f7} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.020025984 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:43.484462 [ 3311 ] {d790e176-1253-4e39-913b-9cb633519fec} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004025582 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:37880) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:43.502217 [ 3314 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:51682). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:43.502261 [ 3316 ] {b9660834-b4be-4e63-8aa4-4bf32316802e} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:48964). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x332c7985 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x332c63d1 in /usr/bin/clickhouse
4. ./build_docker/./src/IO/BufferBase.h:39: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x36b19b94 in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x36afd649 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae59aa in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
12. ? @ 0x7fc159366609 in ?
13. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:43.502511 [ 3316 ] {b9660834-b4be-4e63-8aa4-4bf32316802e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.511932294 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:43.502525 [ 3313 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:33876). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:43.503284 [ 3316 ] {b9660834-b4be-4e63-8aa4-4bf32316802e} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:43.552150 [ 3311 ] {d790e176-1253-4e39-913b-9cb633519fec} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004025582 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:44.914802 [ 3311 ] {03a14a6a-7177-4d4d-bd7d-dd867fb773ca} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.03202608 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:37886) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.369107 [ 3316 ] {e1eb58b9-d04c-4a30-a149-4c6f69fc5bab} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.462440224 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33884) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.386651 [ 3313 ] {91cfe946-16da-4c40-a6ae-4ca4a1182211} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.483908689 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40612) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.386751 [ 3317 ] {0084c1f9-9065-4cb8-bba5-ffd3a4378624} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.484966068 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:38920) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.387473 [ 3312 ] {402f0601-0fca-46d2-a247-bc66798e3d5b} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.112028088 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48980) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.389948 [ 3314 ] {4e0b308d-7ca7-44e6-80ee-fa721a735c5f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.488670025 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51692) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.445116 [ 3311 ] {03a14a6a-7177-4d4d-bd7d-dd867fb773ca} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.03202608 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.478662 [ 3317 ] {0084c1f9-9065-4cb8-bba5-ffd3a4378624} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.484966068 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.483262 [ 3317 ] {0084c1f9-9065-4cb8-bba5-ffd3a4378624} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:45.488501 [ 3312 ] {402f0601-0fca-46d2-a247-bc66798e3d5b} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.112028088 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.490469 [ 3313 ] {91cfe946-16da-4c40-a6ae-4ca4a1182211} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.483908689 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.492750 [ 3316 ] {e1eb58b9-d04c-4a30-a149-4c6f69fc5bab} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.462440224 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.493702 [ 3312 ] {402f0601-0fca-46d2-a247-bc66798e3d5b} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:45.496770 [ 3313 ] {91cfe946-16da-4c40-a6ae-4ca4a1182211} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:45.499578 [ 3316 ] {e1eb58b9-d04c-4a30-a149-4c6f69fc5bab} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:45.502891 [ 3314 ] {4e0b308d-7ca7-44e6-80ee-fa721a735c5f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.488670025 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:45.504549 [ 3314 ] {4e0b308d-7ca7-44e6-80ee-fa721a735c5f} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:46.882045 [ 3311 ] {ea046bbd-9ccc-4937-aa60-3e6101e2985a} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.163201087 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:37598) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.075829 [ 3313 ] {317b0517-3ff0-412b-aea0-653d734407e2} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008025272 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34646) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.078139 [ 3314 ] {5bd94b17-711c-4f0e-aa1f-0db015c718f1} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004025172 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:46982) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.078474 [ 3317 ] {6fcaefd5-0918-4dbe-bb54-c566cd807f31} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.293528819 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44630) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.080090 [ 3312 ] {fdf1b3f7-5670-4c84-ad78-a8d744a04bcb} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004025173 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54228) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.080216 [ 3316 ] {ae77a6fe-8d8c-481d-ae49-678b0ecd453c} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.287295734 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45412) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.082275 [ 3318 ] {9efe6521-3f66-40e3-97b3-7b72d8ffac1f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008025273 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42510) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.112316 [ 3311 ] {ea046bbd-9ccc-4937-aa60-3e6101e2985a} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.163201087 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.129167 [ 3314 ] {5bd94b17-711c-4f0e-aa1f-0db015c718f1} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004025172 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.131718 [ 3314 ] {5bd94b17-711c-4f0e-aa1f-0db015c718f1} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:47.133119 [ 3316 ] {ae77a6fe-8d8c-481d-ae49-678b0ecd453c} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:45412). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x332c7985 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x332c63d1 in /usr/bin/clickhouse
4. ./build_docker/./src/IO/BufferBase.h:39: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x36b19b94 in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x36afd649 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae59aa in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
12. ? @ 0x7fc159366609 in ?
13. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:47.133290 [ 3316 ] {ae77a6fe-8d8c-481d-ae49-678b0ecd453c} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.287295734 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.134304 [ 3316 ] {ae77a6fe-8d8c-481d-ae49-678b0ecd453c} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:47.138837 [ 3318 ] {9efe6521-3f66-40e3-97b3-7b72d8ffac1f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008025273 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.139883 [ 3318 ] {9efe6521-3f66-40e3-97b3-7b72d8ffac1f} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:47.144199 [ 3312 ] {fdf1b3f7-5670-4c84-ad78-a8d744a04bcb} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004025173 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.144517 [ 3312 ] {fdf1b3f7-5670-4c84-ad78-a8d744a04bcb} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:47.155944 [ 3317 ] {6fcaefd5-0918-4dbe-bb54-c566cd807f31} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.293528819 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.156688 [ 3317 ] {6fcaefd5-0918-4dbe-bb54-c566cd807f31} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:47.159216 [ 3313 ] {317b0517-3ff0-412b-aea0-653d734407e2} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008025272 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:47.159942 [ 3313 ] {317b0517-3ff0-412b-aea0-653d734407e2} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:49.329568 [ 3314 ] {f346ff7e-b348-4146-ba56-985b5f1fadab} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.780736161 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53846) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.329611 [ 3313 ] {d985a5f1-47ed-45d7-8eab-9daa408af281} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.79373552 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54234) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.337163 [ 3317 ] {07546b8f-e414-4818-9cf4-56d0d1139030} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.792648242 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44644) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.359579 [ 3316 ] {4e53eb9a-162b-46d2-891e-cb269690d091} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.802389228 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45422) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.370658 [ 1743 ] {9ad071e6-52f6-4908-8b07-c8f7d13cf12e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.820226336 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42514) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.378256 [ 3318 ] {58b6581d-16c7-446f-a0bf-e6c7da4b1bb7} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.837536354 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34656) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.403799 [ 3312 ] {e63a6fc9-2d8c-4b4b-b1b5-42977fed52a3} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.858458962 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:46992) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.460136 [ 3316 ] {4e53eb9a-162b-46d2-891e-cb269690d091} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.802389228 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.460980 [ 1743 ] {9ad071e6-52f6-4908-8b07-c8f7d13cf12e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.820226336 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.470660 [ 3317 ] {07546b8f-e414-4818-9cf4-56d0d1139030} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.792648242 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.472070 [ 3314 ] {f346ff7e-b348-4146-ba56-985b5f1fadab} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.780736161 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.475579 [ 3313 ] {d985a5f1-47ed-45d7-8eab-9daa408af281} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.79373552 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.502993 [ 3311 ] {40ad3bee-4c8c-4f8c-b5da-0a48f3a37159} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.040025905 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:37606) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.511363 [ 3318 ] {58b6581d-16c7-446f-a0bf-e6c7da4b1bb7} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.837536354 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.516919 [ 3314 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:53846). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:49.522531 [ 3318 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:34656). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:49.531270 [ 3312 ] {e63a6fc9-2d8c-4b4b-b1b5-42977fed52a3} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.858458962 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:49.534246 [ 3312 ] {e63a6fc9-2d8c-4b4b-b1b5-42977fed52a3} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:49.543616 [ 1743 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:42514). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:49.548795 [ 3313 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54234). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:49.550385 [ 3316 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:45422). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:49.864388 [ 3311 ] {40ad3bee-4c8c-4f8c-b5da-0a48f3a37159} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.040025905 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.412955 [ 3318 ] {fbf7b94f-3752-44bd-b90c-59572cf434e1} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.240030539 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:37620) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.748397 [ 1305 ] {f64789df-38c8-4e36-83b4-17c44e26b61f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.547814545 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37354) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.750817 [ 3312 ] {627b5c6a-51c5-48b2-a4e4-5320d58966bd} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.542502484 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44652) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.750838 [ 1744 ] {f92f5e43-122d-4574-b7c8-3b261dcae4d5} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.549447176 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34672) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.759772 [ 1743 ] {100c3188-dd34-4e8c-83d6-c5fbb7962374} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.55264411 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:46994) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.761547 [ 3311 ] {f369247a-1c0a-4d7c-80f1-a7c49fbdba29} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.553398501 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45426) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.763445 [ 1745 ] {c8f4da58-0abb-4795-a16f-94cd9b99d2ae} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.144028175 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42516) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.766649 [ 3313 ] {c1c278ad-409b-462a-b762-274e0cf37bf7} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.557968903 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54240) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.778551 [ 3314 ] {a41753a5-ca76-495e-88b4-8ecaab4f87cf} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.561445247 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53852) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.866490 [ 3318 ] {fbf7b94f-3752-44bd-b90c-59572cf434e1} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.240030539 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.884120 [ 1744 ] {f92f5e43-122d-4574-b7c8-3b261dcae4d5} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.549447176 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.906583 [ 1744 ] {f92f5e43-122d-4574-b7c8-3b261dcae4d5} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:51.909589 [ 1745 ] {c8f4da58-0abb-4795-a16f-94cd9b99d2ae} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.144028175 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.922349 [ 1745 ] {c8f4da58-0abb-4795-a16f-94cd9b99d2ae} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:51.923808 [ 1743 ] {100c3188-dd34-4e8c-83d6-c5fbb7962374} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.55264411 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.928266 [ 3312 ] {627b5c6a-51c5-48b2-a4e4-5320d58966bd} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:44652). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x332c7985 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x332c63d1 in /usr/bin/clickhouse
4. ./build_docker/./src/IO/BufferBase.h:39: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x36b19b94 in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x36afd649 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae59aa in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
12. ? @ 0x7fc159366609 in ?
13. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:51.929111 [ 3312 ] {627b5c6a-51c5-48b2-a4e4-5320d58966bd} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.542502484 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.931212 [ 1743 ] {100c3188-dd34-4e8c-83d6-c5fbb7962374} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:51.938735 [ 3312 ] {627b5c6a-51c5-48b2-a4e4-5320d58966bd} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:51.962724 [ 3314 ] {a41753a5-ca76-495e-88b4-8ecaab4f87cf} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:53852). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x332c7985 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x332c63d1 in /usr/bin/clickhouse
4. ./build_docker/./src/IO/BufferBase.h:39: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x36b19b94 in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x36afd649 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae59aa in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
12. ? @ 0x7fc159366609 in ?
13. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:51.968025 [ 3314 ] {a41753a5-ca76-495e-88b4-8ecaab4f87cf} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.561445247 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.976020 [ 3314 ] {a41753a5-ca76-495e-88b4-8ecaab4f87cf} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:51.983574 [ 3311 ] {f369247a-1c0a-4d7c-80f1-a7c49fbdba29} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:45426). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x332c7985 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x332c63d1 in /usr/bin/clickhouse
4. ./build_docker/./src/IO/BufferBase.h:39: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x36b19b94 in /usr/bin/clickhouse
5. ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x36afd649 in /usr/bin/clickhouse
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x36ae59aa in /usr/bin/clickhouse
7. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
11. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
12. ? @ 0x7fc159366609 in ?
13. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:51.983741 [ 3311 ] {f369247a-1c0a-4d7c-80f1-a7c49fbdba29} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.553398501 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.985311 [ 1305 ] {f64789df-38c8-4e36-83b4-17c44e26b61f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.547814545 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.986952 [ 3311 ] {f369247a-1c0a-4d7c-80f1-a7c49fbdba29} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:51.991392 [ 1305 ] {f64789df-38c8-4e36-83b4-17c44e26b61f} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:51.993920 [ 3313 ] {c1c278ad-409b-462a-b762-274e0cf37bf7} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.557968903 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:51.998746 [ 3313 ] {c1c278ad-409b-462a-b762-274e0cf37bf7} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:54.283339 [ 3312 ] {b5941abc-c2fd-4db3-af2a-9932e518d780} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088026591 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55674) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.292111 [ 1305 ] {e4c23a62-24b7-4340-9ce7-0a5f400ccfd7} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.099708755 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45430) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.294023 [ 3311 ] {b9e31154-a9bf-4279-8d8b-f01a3d7fd27a} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.085006678 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34674) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.305657 [ 1743 ] {68924853-47af-4735-b70f-984a8a9e4073} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.118345505 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37364) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.309211 [ 3316 ] {bd7e8bcc-438d-4a23-8322-5751f3a53803} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.103724302 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44656) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.319962 [ 1744 ] {48e0eaf8-553a-4f78-a26d-79f55828caad} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.126525298 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53860) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.352325 [ 3313 ] {4f3e67d8-677f-47b9-b001-49ecc194b058} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.14119984 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47004) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.370134 [ 1745 ] {17f49e3e-9ba7-4383-8675-20d9c33507aa} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.171229547 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54256) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.382042 [ 3314 ] {f0e43c91-c877-4dc7-9c33-1dcd5c445ce0} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.178202491 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42522) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.407540 [ 3316 ] {bd7e8bcc-438d-4a23-8322-5751f3a53803} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.103724302 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.427846 [ 3311 ] {b9e31154-a9bf-4279-8d8b-f01a3d7fd27a} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.085006678 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.434957 [ 1305 ] {e4c23a62-24b7-4340-9ce7-0a5f400ccfd7} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.099708755 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.444915 [ 3313 ] {4f3e67d8-677f-47b9-b001-49ecc194b058} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.14119984 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.445806 [ 3312 ] {b5941abc-c2fd-4db3-af2a-9932e518d780} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088026591 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.448546 [ 1744 ] {48e0eaf8-553a-4f78-a26d-79f55828caad} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.126525298 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.454635 [ 1743 ] {68924853-47af-4735-b70f-984a8a9e4073} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.118345505 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.464467 [ 3318 ] {a70286f3-934f-4ef0-a7e9-80820290a9fb} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008024649 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:37624) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.475473 [ 1745 ] {17f49e3e-9ba7-4383-8675-20d9c33507aa} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.171229547 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.476381 [ 1743 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:37364). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:54.488540 [ 1744 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:53860). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:54.489472 [ 3313 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:47004). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:54.491805 [ 3311 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:34674). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:54.507512 [ 1745 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54256). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:54.508335 [ 3316 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:44656). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:54.509351 [ 1305 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:45430). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x332c4331 in /usr/bin/clickhouse
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x332c29db in /usr/bin/clickhouse
4. ./build_docker/./src/IO/ReadBuffer.h:67: DB::TCPHandler::runImpl() @ 0x36ae1892 in /usr/bin/clickhouse
5. ./build_docker/./src/Server/TCPHandler.cpp:2079: DB::TCPHandler::run() @ 0x36b1a58d in /usr/bin/clickhouse
6. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x3e971eef in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x3e972be5 in /usr/bin/clickhouse
8. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x3ee3699c in /usr/bin/clickhouse
9. ./build_docker/./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x3ee2fecb in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.19 19:13:54.520730 [ 3314 ] {f0e43c91-c877-4dc7-9c33-1dcd5c445ce0} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.178202491 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x36be1e6e in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long) @ 0x36be1616 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:95: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36c0e2a4 in /usr/bin/clickhouse
7. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
9. ? @ 0x7fc159366609 in ?
10. clone @ 0x7fc15928b353 in ?

2024.03.19 19:13:54.523355 [ 3314 ] {f0e43c91-c877-4dc7-9c33-1dcd5c445ce0} <Warning> TCPHandler: Client has gone away.
2024.03.19 19:13:54.639798 [ 3318 ] {a70286f3-934f-4ef0-a7e9-80820290a9fb} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008024649 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x3ed0f6ac in /usr/bin/clickhouse
1. ./build_docker/./src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x203a102a in /usr/bin/clickhouse
2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x3099d505 in /usr/bin/clickhouse
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:110: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x3099c77e in /usr/bin/clickhouse
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:0: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x3099a327 in /usr/bin/clickhouse
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::ExecutionThreadContext::executeTask() @ 0x36c03e48 in /usr/bin/clickhouse
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x36be31dd in /usr/bin/clickhouse
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:345: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x36be6c7f in /usr/bin/clickhouse
8. ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:0: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x2061770f in /usr/bin/clickhouse
9. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x206241c0 in /usr/bin/clickhouse
10. ? @ 0x7fc159366609 in ?
11. clone @ 0x7fc15928b353 in ?