2024.03.27 10:27:56.375947 [ 333 ] {} <Warning> Context: Server was built in debug mode. It will work slowly.
2024.03.27 10:27:56.376407 [ 333 ] {} <Warning> Context: Server logging level is set to 'test' and performance is degraded. This cannot be used in production.
2024.03.27 10:27:56.379156 [ 333 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.03.27 10:27:58.676884 [ 333 ] {} <Warning> KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper.
2024.03.27 10:27:58.677548 [ 333 ] {} <Warning> KeeperLogStore: Removing all changelogs
2024.03.27 10:27:58.678792 [ 333 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.03.27 10:27:58.678880 [ 333 ] {} <Fatal> KeeperServer: Can't start NON-SECURE keeper server in FIPS mode, please check the config.
2024.03.27 10:27:58.682052 [ 333 ] {} <Warning> KeeperStateManager: No state was read
2024.03.27 10:27:58.682135 [ 333 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.03.27 10:27:58.682209 [ 333 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.03.27 10:27:58.733303 [ 567 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.03.27 10:27:58.792728 [ 333 ] {} <Warning> Access(local directory): File /var/lib/clickhouse/access/users.list doesn't exist
2024.03.27 10:27:58.792849 [ 333 ] {} <Warning> Access(local directory): Recovering lists in directory /var/lib/clickhouse/access/
2024.03.27 10:27:58.975552 [ 333 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.03.27 10:28:28.431122 [ 333 ] {} <Warning> KeeperLogStore: Changelog is shut down
2024.03.27 10:28:31.213408 [ 722 ] {} <Warning> Context: Server was built in debug mode. It will work slowly.
2024.03.27 10:28:31.213723 [ 722 ] {} <Warning> Context: Server logging level is set to 'test' and performance is degraded. This cannot be used in production.
2024.03.27 10:28:31.215698 [ 722 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.03.27 10:28:33.514692 [ 722 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.03.27 10:28:33.514841 [ 722 ] {} <Fatal> KeeperServer: Can't start NON-SECURE keeper server in FIPS mode, please check the config.
2024.03.27 10:28:33.520317 [ 722 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.03.27 10:28:33.520438 [ 722 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.03.27 10:28:33.572210 [ 940 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.03.27 10:28:33.899431 [ 1018 ] {} <Error> Container already exists, returning the existing container: std::exception. Code: 1001, type: Azure::Storage::StorageException, e.what() = 409 The specified container already exists.
The specified container already exists.
RequestId:8fcb7257-d674-4ad4-8c46-56e76d4142ed
Time:2024-03-27T14:28:33.748Z
Request ID: 8fcb7257-d674-4ad4-8c46-56e76d4142ed, Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/contrib/llvm-project/libcxx/src/support/runtime/stdexcept_default.ipp:33: std::runtime_error::runtime_error(String const&) @ 0x35a2bc7d in /usr/bin/clickhouse
3. /build/contrib/azure/sdk/core/azure-core/inc/azure/core/exception.hpp:78: Azure::Core::RequestFailedException::RequestFailedException(String const&) @ 0x2d53afe1 in /usr/bin/clickhouse
4. /build/contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/storage_exception.hpp:26: Azure::Storage::StorageException::StorageException(String const&) @ 0x2d618241 in /usr/bin/clickhouse
5. /build/contrib/azure/sdk/storage/azure-storage-common/src/storage_exception.cpp:144: Azure::Storage::StorageException::CreateFromResponse(std::unique_ptr<Azure::Core::Http::RawResponse, std::default_delete<Azure::Core::Http::RawResponse>>) @ 0x2d615e58 in /usr/bin/clickhouse
6. /build/contrib/azure/sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/protocol/blob_rest_client.hpp:4760: Azure::Storage::Blobs::_detail::BlobRestClient::BlobContainer::Create(Azure::Core::Http::_internal::HttpPipeline&, Azure::Core::Url const&, Azure::Storage::Blobs::_detail::BlobRestClient::BlobContainer::CreateBlobContainerOptions const&, Azure::Core::Context const&) @ 0x2d592de4 in /usr/bin/clickhouse
7. /build/contrib/azure/sdk/storage/azure-storage-blobs/src/blob_container_client.cpp:147: Azure::Storage::Blobs::BlobContainerClient::Create(Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const @ 0x2d590b4c in /usr/bin/clickhouse
8. /build/contrib/azure/sdk/storage/azure-storage-blobs/src/blob_service_client.cpp:235: Azure::Storage::Blobs::BlobServiceClient::CreateBlobContainer(String const&, Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const @ 0x2d5b85c6 in /usr/bin/clickhouse
9. /build/src/Disks/ObjectStorages/AzureBlobStorage/AzureBlobStorageAuth.cpp:132: DB::getAzureBlobContainerClient(Poco::Util::AbstractConfiguration const&, String const&) @ 0x29c9f41b in /usr/bin/clickhouse
10. /build/src/Disks/ObjectStorages/AzureBlobStorage/registerDiskAzureBlobStorage.cpp:31: DB::registerDiskAzureBlobStorage(DB::DiskFactory&, bool)::$_0::operator()(String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) const @ 0x29c9cd0d in /usr/bin/clickhouse
11. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::registerDiskAzureBlobStorage(DB::DiskFactory&, bool)::$_0&>()(std::declval<String const&>(), std::declval<Poco::Util::AbstractConfiguration const&>(), std::declval<String const&>(), std::declval<std::shared_ptr<DB::Context const>>(), std::declval<std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&>())) std::__invoke[abi:v15000]<DB::registerDiskAzureBlobStorage(DB::DiskFactory&, bool)::$_0&, String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&>(DB::registerDiskAzureBlobStorage(DB::DiskFactory&, bool)::$_0&, String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>&&, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) @ 0x29c9cc17 in /usr/bin/clickhouse
12. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:470: std::shared_ptr<DB::IDisk> std::__invoke_void_return_wrapper<std::shared_ptr<DB::IDisk>, false>::__call<DB::registerDiskAzureBlobStorage(DB::DiskFactory&, bool)::$_0&, String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&>(DB::registerDiskAzureBlobStorage(DB::DiskFactory&, bool)::$_0&, String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>&&, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) @ 0x29c9cb7d in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::registerDiskAzureBlobStorage(DB::DiskFactory&, bool)::$_0, std::shared_ptr<DB::IDisk> (String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&)>::operator()[abi:v15000](String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>&&, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) @ 0x29c9cb1d in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: std::shared_ptr<DB::IDisk> std::__function::__policy_invoker<std::shared_ptr<DB::IDisk> (String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerDiskAzureBlobStorage(DB::DiskFactory&, bool)::$_0, std::shared_ptr<DB::IDisk> (String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&)>>(std::__function::__policy_storage const*, String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>&&, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) @ 0x29c9cac5 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<std::shared_ptr<DB::IDisk> (String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&)>::operator()[abi:v15000](String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>&&, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) const @ 0x29bd49de in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<std::shared_ptr<DB::IDisk> (String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&)>::operator()(String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) const @ 0x29bd20a1 in /usr/bin/clickhouse
17. /build/src/Disks/DiskFactory.cpp:37: DB::DiskFactory::create(String const&, Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>, std::map<String, std::shared_ptr<DB::IDisk>, std::less<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::IDisk>>>> const&) const @ 0x29bd1dfd in /usr/bin/clickhouse
18. /build/src/Disks/DiskSelector.cpp:49: DB::DiskSelector::initialize(Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::Context const>) @ 0x29f1901c in /usr/bin/clickhouse
19. /build/src/Interpreters/Context.cpp:3123: DB::Context::getDiskSelector(std::lock_guard<std::mutex>&) const @ 0x29e83a00 in /usr/bin/clickhouse
20. /build/src/Interpreters/Context.cpp:3136: DB::Context::getStoragePolicySelector(std::lock_guard<std::mutex>&) const @ 0x29e71100 in /usr/bin/clickhouse
21. /build/src/Interpreters/Context.cpp:3068: DB::Context::getStoragePolicy(String const&) const @ 0x29e83de0 in /usr/bin/clickhouse
22. /build/src/Storages/MergeTree/MergeTreeData.cpp:400: DB::MergeTreeData::getStoragePolicy() const @ 0x2c0bc151 in /usr/bin/clickhouse
23. /build/src/Storages/MergeTree/MergeTreeData.h:912: DB::MergeTreeData::getDisks() const @ 0x2bf7150b in /usr/bin/clickhouse
24. /build/src/Storages/MergeTree/MergeTreeData.cpp:229: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x2c0b7928 in /usr/bin/clickhouse
25. /build/src/Storages/StorageMergeTree.cpp:110: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, bool, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool) @ 0x2c56b0df in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::StorageMergeTree* std::construct_at[abi:v15000]<DB::StorageMergeTree, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool const&, DB::StorageMergeTree*>(DB::StorageMergeTree*, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, bool const&) @ 0x2c56aa9a in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:302: void std::allocator_traits<std::allocator<DB::StorageMergeTree>>::construct[abi:v15000]<DB::StorageMergeTree, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool const&, void, void>(std::allocator<DB::StorageMergeTree>&, DB::StorageMergeTree*, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, bool const&) @ 0x2c56a795 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:292: std::__shared_ptr_emplace<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>>::__shared_ptr_emplace[abi:v15000]<DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool const&>(std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, bool const&) @ 0x2c56a423 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:953: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool const&, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, bool const&) @ 0x2c56a235 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: std::shared_ptr<DB::StorageMergeTree> std::make_shared[abi:v15000]<DB::StorageMergeTree, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool const&, void>(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, bool const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, bool const&) @ 0x2c5689db in /usr/bin/clickhouse
31. /build/src/Storages/MergeTree/registerStorageMergeTree.cpp:702: DB::create(DB::StorageFactory::Arguments const&) @ 0x2c566997 in /usr/bin/clickhouse
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:28:34.806379 [ 722 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.03.27 10:32:56.049817 [ 1458 ] {3f997c55-5d0c-420c-9332-1d79de690b3e} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery3: While processing 1 = ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery3). (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:51104) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x299250d9 in /usr/bin/clickhouse
8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:168: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x2cf16423 in /usr/bin/clickhouse
9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1637: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x2cf23cc6 in /usr/bin/clickhouse
10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x2ced62ec in /usr/bin/clickhouse
11. /build/src/Processors/QueryPlan/QueryPlan.cpp:187: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x2cefb6df in /usr/bin/clickhouse
12. /build/src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x2ae7f60b in /usr/bin/clickhouse
13. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:188: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d7888 in /usr/bin/clickhouse
14. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:67: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d6cc1 in /usr/bin/clickhouse
15. /build/src/Interpreters/InDepthNodeVisitor.h:71: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x2b243040 in /usr/bin/clickhouse
16. /build/src/Interpreters/InDepthNodeVisitor.h:61: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b243221 in /usr/bin/clickhouse
17. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
18. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
19. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse
20. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
21. /build/src/Interpreters/InDepthNodeVisitor.h:33: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visit(std::shared_ptr<DB::IAST>&) @ 0x2b242ea5 in /usr/bin/clickhouse
22. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:318: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d933d in /usr/bin/clickhouse
23. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:69: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d6cf7 in /usr/bin/clickhouse
24. /build/src/Interpreters/InDepthNodeVisitor.h:71: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x2b243040 in /usr/bin/clickhouse
25. /build/src/Interpreters/InDepthNodeVisitor.h:61: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b243221 in /usr/bin/clickhouse
26. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
27. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
28. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse
29. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
30. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
31. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse

2024.03.27 10:32:56.051056 [ 1458 ] {3f997c55-5d0c-420c-9332-1d79de690b3e} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery3: While processing 1 = ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery3). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x299250d9 in /usr/bin/clickhouse
8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:168: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x2cf16423 in /usr/bin/clickhouse
9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1637: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x2cf23cc6 in /usr/bin/clickhouse
10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x2ced62ec in /usr/bin/clickhouse
11. /build/src/Processors/QueryPlan/QueryPlan.cpp:187: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x2cefb6df in /usr/bin/clickhouse
12. /build/src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x2ae7f60b in /usr/bin/clickhouse
13. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:188: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d7888 in /usr/bin/clickhouse
14. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:67: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d6cc1 in /usr/bin/clickhouse
15. /build/src/Interpreters/InDepthNodeVisitor.h:71: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x2b243040 in /usr/bin/clickhouse
16. /build/src/Interpreters/InDepthNodeVisitor.h:61: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b243221 in /usr/bin/clickhouse
17. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
18. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
19. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse
20. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
21. /build/src/Interpreters/InDepthNodeVisitor.h:33: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visit(std::shared_ptr<DB::IAST>&) @ 0x2b242ea5 in /usr/bin/clickhouse
22. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:318: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d933d in /usr/bin/clickhouse
23. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:69: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d6cf7 in /usr/bin/clickhouse
24. /build/src/Interpreters/InDepthNodeVisitor.h:71: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x2b243040 in /usr/bin/clickhouse
25. /build/src/Interpreters/InDepthNodeVisitor.h:61: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b243221 in /usr/bin/clickhouse
26. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
27. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
28. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse
29. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
30. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
31. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse

2024.03.27 10:32:56.200950 [ 1458 ] {6038dfed-749a-4243-b3c8-2d47a57c0332} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:51104) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x299250d9 in /usr/bin/clickhouse
8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:168: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x2cf16423 in /usr/bin/clickhouse
9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1637: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x2cf23cc6 in /usr/bin/clickhouse
10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x2ced62ec in /usr/bin/clickhouse
11. /build/src/Processors/QueryPlan/QueryPlan.cpp:187: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x2cefb6df in /usr/bin/clickhouse
12. /build/src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x2ae7f60b in /usr/bin/clickhouse
13. /build/src/Interpreters/executeQuery.cpp:713: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x2b351757 in /usr/bin/clickhouse
14. /build/src/Interpreters/executeQuery.cpp:1160: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x2b34d424 in /usr/bin/clickhouse
15. /build/src/Server/TCPHandler.cpp:420: DB::TCPHandler::runImpl() @ 0x2c7a93ad in /usr/bin/clickhouse
16. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
17. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
18. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
19. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
20. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
21. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
22. ? @ 0x7f4161c1e609 in ?
23. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:32:56.202278 [ 1458 ] {6038dfed-749a-4243-b3c8-2d47a57c0332} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x299250d9 in /usr/bin/clickhouse
8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:168: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x2cf16423 in /usr/bin/clickhouse
9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1637: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x2cf23cc6 in /usr/bin/clickhouse
10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x2ced62ec in /usr/bin/clickhouse
11. /build/src/Processors/QueryPlan/QueryPlan.cpp:187: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x2cefb6df in /usr/bin/clickhouse
12. /build/src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x2ae7f60b in /usr/bin/clickhouse
13. /build/src/Interpreters/executeQuery.cpp:713: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x2b351757 in /usr/bin/clickhouse
14. /build/src/Interpreters/executeQuery.cpp:1160: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x2b34d424 in /usr/bin/clickhouse
15. /build/src/Server/TCPHandler.cpp:420: DB::TCPHandler::runImpl() @ 0x2c7a93ad in /usr/bin/clickhouse
16. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
17. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
18. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
19. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
20. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
21. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
22. ? @ 0x7f4161c1e609 in ?
23. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:32:56.479805 [ 1458 ] {c7414372-94d3-4dce-ae18-f23680cb6a10} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeThread. (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:51104) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x2990204a in /usr/bin/clickhouse
8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
28. ? @ 0x7f4161c1e609 in ?
29. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:32:56.496723 [ 1458 ] {c7414372-94d3-4dce-ae18-f23680cb6a10} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 15.00 million, current rows: 17.76 million: While executing MergeTreeThread. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x2990204a in /usr/bin/clickhouse
8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
28. ? @ 0x7f4161c1e609 in ?
29. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:32:56.686618 [ 1458 ] {a93b59d2-0c7f-48f7-a73c-f4e84402728b} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery6: While processing CounterID > ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery6). (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:51104) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID > (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x299250d9 in /usr/bin/clickhouse
8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:168: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x2cf16423 in /usr/bin/clickhouse
9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1637: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x2cf23cc6 in /usr/bin/clickhouse
10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x2ced62ec in /usr/bin/clickhouse
11. /build/src/Processors/QueryPlan/QueryPlan.cpp:187: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x2cefb6df in /usr/bin/clickhouse
12. /build/src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x2ae7f60b in /usr/bin/clickhouse
13. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:188: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d7888 in /usr/bin/clickhouse
14. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:67: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d6cc1 in /usr/bin/clickhouse
15. /build/src/Interpreters/InDepthNodeVisitor.h:71: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x2b243040 in /usr/bin/clickhouse
16. /build/src/Interpreters/InDepthNodeVisitor.h:61: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b243221 in /usr/bin/clickhouse
17. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
18. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
19. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse
20. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
21. /build/src/Interpreters/InDepthNodeVisitor.h:33: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visit(std::shared_ptr<DB::IAST>&) @ 0x2b242ea5 in /usr/bin/clickhouse
22. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:318: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d933d in /usr/bin/clickhouse
23. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:69: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d6cf7 in /usr/bin/clickhouse
24. /build/src/Interpreters/InDepthNodeVisitor.h:71: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x2b243040 in /usr/bin/clickhouse
25. /build/src/Interpreters/InDepthNodeVisitor.h:61: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b243221 in /usr/bin/clickhouse
26. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
27. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
28. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse
29. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
30. /build/src/Interpreters/InDepthNodeVisitor.h:33: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visit(std::shared_ptr<DB::IAST>&) @ 0x2b242ea5 in /usr/bin/clickhouse
31. /build/src/Interpreters/TreeRewriter.cpp:459: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x2b233055 in /usr/bin/clickhouse

2024.03.27 10:32:56.688037 [ 1458 ] {a93b59d2-0c7f-48f7-a73c-f4e84402728b} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.88 million: While processing (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery6: While processing CounterID > ((SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) AS _subquery6). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x299250d9 in /usr/bin/clickhouse
8. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:168: DB::ReadFromMergeTree::AnalysisResult::checkLimits(DB::Settings const&, DB::SelectQueryInfo const&) const @ 0x2cf16423 in /usr/bin/clickhouse
9. /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1637: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x2cf23cc6 in /usr/bin/clickhouse
10. /build/src/Processors/QueryPlan/ISourceStep.cpp:20: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x2ced62ec in /usr/bin/clickhouse
11. /build/src/Processors/QueryPlan/QueryPlan.cpp:187: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x2cefb6df in /usr/bin/clickhouse
12. /build/src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379: DB::InterpreterSelectWithUnionQuery::execute() @ 0x2ae7f60b in /usr/bin/clickhouse
13. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:188: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTSubquery const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d7888 in /usr/bin/clickhouse
14. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:67: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d6cc1 in /usr/bin/clickhouse
15. /build/src/Interpreters/InDepthNodeVisitor.h:71: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x2b243040 in /usr/bin/clickhouse
16. /build/src/Interpreters/InDepthNodeVisitor.h:61: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b243221 in /usr/bin/clickhouse
17. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
18. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
19. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse
20. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
21. /build/src/Interpreters/InDepthNodeVisitor.h:33: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visit(std::shared_ptr<DB::IAST>&) @ 0x2b242ea5 in /usr/bin/clickhouse
22. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:318: DB::ExecuteScalarSubqueriesMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d933d in /usr/bin/clickhouse
23. /build/src/Interpreters/ExecuteScalarSubqueriesVisitor.cpp:69: DB::ExecuteScalarSubqueriesMatcher::visit(std::shared_ptr<DB::IAST>&, DB::ExecuteScalarSubqueriesMatcher::Data&) @ 0x2b2d6cf7 in /usr/bin/clickhouse
24. /build/src/Interpreters/InDepthNodeVisitor.h:71: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::doVisit(std::shared_ptr<DB::IAST>&) @ 0x2b243040 in /usr/bin/clickhouse
25. /build/src/Interpreters/InDepthNodeVisitor.h:61: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b243221 in /usr/bin/clickhouse
26. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
27. /build/src/Interpreters/InDepthNodeVisitor.h:83: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitChildren<false>(std::shared_ptr<DB::IAST>&) @ 0x2b2432c6 in /usr/bin/clickhouse
28. /build/src/Interpreters/InDepthNodeVisitor.h:65: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImplMain<false>(std::shared_ptr<DB::IAST>&) @ 0x2b24322e in /usr/bin/clickhouse
29. /build/src/Interpreters/InDepthNodeVisitor.h:53: void DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visitImpl<false>(std::shared_ptr<DB::IAST>&) @ 0x2b242fca in /usr/bin/clickhouse
30. /build/src/Interpreters/InDepthNodeVisitor.h:33: DB::InDepthNodeVisitor<DB::ExecuteScalarSubqueriesMatcher, true, false, std::shared_ptr<DB::IAST>>::visit(std::shared_ptr<DB::IAST>&) @ 0x2b242ea5 in /usr/bin/clickhouse
31. /build/src/Interpreters/TreeRewriter.cpp:459: DB::(anonymous namespace)::executeScalarSubqueries(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context const>, unsigned long, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, std::map<String, DB::Block, std::less<String>, std::allocator<std::pair<String const, DB::Block>>>&, bool, bool) @ 0x2b233055 in /usr/bin/clickhouse

2024.03.27 10:32:56.874291 [ 1458 ] {bb94cfcd-8d45-4a0d-a917-c1d24443f6e3} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeThread. (TOO_MANY_ROWS) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:51104) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() FROM test.hits WHERE CounterID < (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x2990204a in /usr/bin/clickhouse
8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
28. ? @ 0x7f4161c1e609 in ?
29. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:32:56.890459 [ 1458 ] {bb94cfcd-8d45-4a0d-a917-c1d24443f6e3} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 10.00 million, current rows: 13.12 million: While executing MergeTreeThread. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x29925268 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924d8c in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x2990204a in /usr/bin/clickhouse
8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
28. ? @ 0x7f4161c1e609 in ?
29. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:47.286641 [ 1259 ] {37ec6378-08cd-4768-a253-a98c64fd6fd7} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.85 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:43570) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x29925448 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924f67 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x2990204a in /usr/bin/clickhouse
8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
28. ? @ 0x7f4161c1e609 in ?
29. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:47.294746 [ 1259 ] {37ec6378-08cd-4768-a253-a98c64fd6fd7} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 574.85 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x29925448 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924f67 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x2990204a in /usr/bin/clickhouse
8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
28. ? @ 0x7f4161c1e609 in ?
29. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:48.959439 [ 1632 ] {f462358a-2e41-4c4a-86f5-761374f05f35} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.036788448 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:59670) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:49.095975 [ 729 ] {af67757f-b8f3-40dd-a15b-f04818add1f8} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60358). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60358) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2021: DB::TCPHandler::sendEndOfStream() @ 0x2c7b2963 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:501: DB::TCPHandler::runImpl() @ 0x2c7a9e72 in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
16. ? @ 0x7f4161c1e609 in ?
17. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:49.176530 [ 729 ] {af67757f-b8f3-40dd-a15b-f04818add1f8} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60358). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:40:49.177018 [ 1632 ] {f462358a-2e41-4c4a-86f5-761374f05f35} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.036788448 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:49.177156 [ 729 ] {af67757f-b8f3-40dd-a15b-f04818add1f8} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60358). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2021: DB::TCPHandler::sendEndOfStream() @ 0x2c7b2963 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:501: DB::TCPHandler::runImpl() @ 0x2c7a9e72 in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
16. ? @ 0x7f4161c1e609 in ?
17. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:49.177836 [ 729 ] {af67757f-b8f3-40dd-a15b-f04818add1f8} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:40:49.269436 [ 1259 ] {bd2ada83-b2e5-4dc6-8be3-73693eedce42} <Error> executeQuery: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 575.37 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:43570) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x29925448 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924f67 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x2990204a in /usr/bin/clickhouse
8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
28. ? @ 0x7f4161c1e609 in ?
29. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:49.283029 [ 1259 ] {bd2ada83-b2e5-4dc6-8be3-73693eedce42} <Error> TCPHandler: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: 572.20 MiB, current bytes: 575.37 MiB: While executing MergeTreeThread. (TOO_MANY_BYTES), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x29925448 in /usr/bin/clickhouse
6. /build/src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x29924f67 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ReadProgressCallback.cpp:97: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x2990204a in /usr/bin/clickhouse
8. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
28. ? @ 0x7f4161c1e609 in ?
29. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:51.798458 [ 1259 ] {07bcd6cd-11f7-41f3-828a-20ceb0785fd7} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088273204 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:59674) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:52.030814 [ 1458 ] {f27a2283-8a3d-4641-aa06-75713f4dd3e6} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:33770). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33770) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:52.121459 [ 729 ] {e1b00a27-479a-4323-9871-d0a7a94be0ae} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60366). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60366) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:52.121828 [ 1259 ] {07bcd6cd-11f7-41f3-828a-20ceb0785fd7} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.088273204 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:52.485738 [ 1458 ] {f27a2283-8a3d-4641-aa06-75713f4dd3e6} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:33770). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:40:52.486516 [ 1458 ] {f27a2283-8a3d-4641-aa06-75713f4dd3e6} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:33770). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:52.499367 [ 1458 ] {f27a2283-8a3d-4641-aa06-75713f4dd3e6} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:40:52.789322 [ 729 ] {e1b00a27-479a-4323-9871-d0a7a94be0ae} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60366). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:40:52.789965 [ 729 ] {e1b00a27-479a-4323-9871-d0a7a94be0ae} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60366). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:52.796742 [ 729 ] {e1b00a27-479a-4323-9871-d0a7a94be0ae} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:40:54.741585 [ 1259 ] {a150c1df-a0fe-44da-8a11-d28533883335} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.122416118 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:59688) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:54.913901 [ 2910 ] {d3743de5-18e1-45a2-8de4-47ed5fa5e35a} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60368). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60368) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:54.934432 [ 1458 ] {e6495095-54b9-444a-92c1-44dc8fa0d6bb} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004091434 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50570) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:54.953378 [ 729 ] {1e89c1a0-68d0-49e9-9eaa-2bce4b751005} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008058705 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33786) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:54.990187 [ 1259 ] {a150c1df-a0fe-44da-8a11-d28533883335} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.122416118 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:55.149534 [ 2910 ] {d3743de5-18e1-45a2-8de4-47ed5fa5e35a} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60368). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:40:55.150124 [ 2910 ] {d3743de5-18e1-45a2-8de4-47ed5fa5e35a} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60368). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:55.161711 [ 2910 ] {d3743de5-18e1-45a2-8de4-47ed5fa5e35a} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:40:55.243573 [ 729 ] {1e89c1a0-68d0-49e9-9eaa-2bce4b751005} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008058705 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:55.244903 [ 729 ] {1e89c1a0-68d0-49e9-9eaa-2bce4b751005} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:40:55.300839 [ 1458 ] {e6495095-54b9-444a-92c1-44dc8fa0d6bb} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.004091434 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:55.303197 [ 1458 ] {e6495095-54b9-444a-92c1-44dc8fa0d6bb} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:40:57.659838 [ 1259 ] {d1866c98-d1be-4a72-a54e-da09f92d1886} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.173571698 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:44850) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:57.729168 [ 1458 ] {c29d81a9-f866-47e3-b51e-1d11733ab9ec} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:56072). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56072) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:57.736129 [ 2910 ] {46221fce-e9c2-40cc-b061-049d0c896f7b} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:37538). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37538) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:57.744130 [ 729 ] {f55a8c59-0a53-4992-962b-a517b1a63bc1} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:53522). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53522) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:57.819487 [ 2914 ] {79fcb7b1-63fa-442a-9b40-8cc25812395c} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56222). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56222) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:57.823884 [ 2910 ] {46221fce-e9c2-40cc-b061-049d0c896f7b} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:37538). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:40:57.825258 [ 2910 ] {46221fce-e9c2-40cc-b061-049d0c896f7b} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:37538). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:57.839134 [ 2910 ] {46221fce-e9c2-40cc-b061-049d0c896f7b} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:40:57.937095 [ 729 ] {f55a8c59-0a53-4992-962b-a517b1a63bc1} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:53522). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:40:57.937883 [ 729 ] {f55a8c59-0a53-4992-962b-a517b1a63bc1} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:53522). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:57.938708 [ 729 ] {f55a8c59-0a53-4992-962b-a517b1a63bc1} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:40:57.955586 [ 1259 ] {d1866c98-d1be-4a72-a54e-da09f92d1886} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.173571698 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:58.011442 [ 1458 ] {c29d81a9-f866-47e3-b51e-1d11733ab9ec} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:56072). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:40:58.013143 [ 1458 ] {c29d81a9-f866-47e3-b51e-1d11733ab9ec} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:56072). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:58.018461 [ 1458 ] {c29d81a9-f866-47e3-b51e-1d11733ab9ec} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:40:58.132607 [ 2914 ] {79fcb7b1-63fa-442a-9b40-8cc25812395c} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56222). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:40:58.133407 [ 2914 ] {79fcb7b1-63fa-442a-9b40-8cc25812395c} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56222). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:40:58.137794 [ 2914 ] {79fcb7b1-63fa-442a-9b40-8cc25812395c} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:00.825030 [ 2910 ] {c7da8486-8937-4ff9-9b9f-c56cc21ba864} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.021461257 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56086) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:00.949699 [ 1259 ] {7fd817c4-d30f-4425-a857-4de928fb7e6b} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.076049267 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:44864) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:01.065323 [ 2914 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:45384). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x2ab9546c in /usr/bin/clickhouse
6. /build/src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x2ab94828 in /usr/bin/clickhouse
7. /build/src/IO/ReadBuffer.h:66: DB::ReadBuffer::next() @ 0x197399fc in /usr/bin/clickhouse
8. /build/src/IO/ReadBuffer.h:100: DB::TCPHandler::runImpl() @ 0x2c7a7fa6 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
10. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
12. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
15. ? @ 0x7f4161c1e609 in ?
16. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:01.087237 [ 729 ] {02c29fe3-dddc-4164-a697-bc6c97ead852} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:53526). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53526) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2021: DB::TCPHandler::sendEndOfStream() @ 0x2c7b2963 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:501: DB::TCPHandler::runImpl() @ 0x2c7a9e72 in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
16. ? @ 0x7f4161c1e609 in ?
17. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:01.091077 [ 729 ] {02c29fe3-dddc-4164-a697-bc6c97ead852} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:53526). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:01.091342 [ 1458 ] {060b7730-6bcd-49e2-8b58-2febffab56b1} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56226). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56226) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2021: DB::TCPHandler::sendEndOfStream() @ 0x2c7b2963 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:501: DB::TCPHandler::runImpl() @ 0x2c7a9e72 in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
16. ? @ 0x7f4161c1e609 in ?
17. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:01.091727 [ 729 ] {02c29fe3-dddc-4164-a697-bc6c97ead852} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:53526). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2021: DB::TCPHandler::sendEndOfStream() @ 0x2c7b2963 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:501: DB::TCPHandler::runImpl() @ 0x2c7a9e72 in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
16. ? @ 0x7f4161c1e609 in ?
17. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:01.092933 [ 729 ] {02c29fe3-dddc-4164-a697-bc6c97ead852} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:01.094855 [ 1458 ] {060b7730-6bcd-49e2-8b58-2febffab56b1} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56226). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:01.095627 [ 1458 ] {060b7730-6bcd-49e2-8b58-2febffab56b1} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56226). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2021: DB::TCPHandler::sendEndOfStream() @ 0x2c7b2963 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:501: DB::TCPHandler::runImpl() @ 0x2c7a9e72 in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
16. ? @ 0x7f4161c1e609 in ?
17. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:01.100373 [ 1458 ] {060b7730-6bcd-49e2-8b58-2febffab56b1} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:01.101092 [ 2916 ] {9e0ea767-8b4a-48fd-a0d5-13b3b3aed605} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:37550). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37550) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2021: DB::TCPHandler::sendEndOfStream() @ 0x2c7b2963 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:501: DB::TCPHandler::runImpl() @ 0x2c7a9e72 in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
16. ? @ 0x7f4161c1e609 in ?
17. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:01.106149 [ 2916 ] {9e0ea767-8b4a-48fd-a0d5-13b3b3aed605} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:37550). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:01.107527 [ 2916 ] {9e0ea767-8b4a-48fd-a0d5-13b3b3aed605} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:37550). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2021: DB::TCPHandler::sendEndOfStream() @ 0x2c7b2963 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:501: DB::TCPHandler::runImpl() @ 0x2c7a9e72 in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
16. ? @ 0x7f4161c1e609 in ?
17. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:01.112247 [ 2916 ] {9e0ea767-8b4a-48fd-a0d5-13b3b3aed605} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:01.217982 [ 2910 ] {c7da8486-8937-4ff9-9b9f-c56cc21ba864} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.021461257 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:01.219766 [ 2910 ] {c7da8486-8937-4ff9-9b9f-c56cc21ba864} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:01.431365 [ 1259 ] {7fd817c4-d30f-4425-a857-4de928fb7e6b} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.076049267 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.000297 [ 1259 ] {90d2fa2c-85f8-4ef5-a324-bd00ed31f7be} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.119392164 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:44878) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.191923 [ 2914 ] {07c8d324-34e8-4f5d-9883-97720737272c} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:37558). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37558) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.193499 [ 729 ] {85375e91-85f4-4476-9151-9ab95e6b6167} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56230). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56230) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.213716 [ 2918 ] {e5d7efc0-5d45-44c3-9e59-026234e42334} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:34504). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34504) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.216908 [ 2916 ] {a8115d6a-0fdf-49c1-82d5-2db29ab278db} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:56090). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56090) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.224584 [ 1458 ] {e08b52f4-0676-4a7f-933c-9c68f35fdc82} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:53528). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53528) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.250372 [ 2910 ] {ce628190-fba6-4810-937a-da520c439f18} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:45398). (NETWORK_ERROR) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45398) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.272328 [ 2914 ] {07c8d324-34e8-4f5d-9883-97720737272c} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:37558). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:04.273432 [ 2914 ] {07c8d324-34e8-4f5d-9883-97720737272c} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:37558). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.276081 [ 2916 ] {a8115d6a-0fdf-49c1-82d5-2db29ab278db} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:56090). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:04.276671 [ 2916 ] {a8115d6a-0fdf-49c1-82d5-2db29ab278db} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:56090). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.286450 [ 2914 ] {07c8d324-34e8-4f5d-9883-97720737272c} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:04.288149 [ 1259 ] {90d2fa2c-85f8-4ef5-a324-bd00ed31f7be} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.119392164 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.295646 [ 2916 ] {a8115d6a-0fdf-49c1-82d5-2db29ab278db} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:04.323803 [ 2910 ] {ce628190-fba6-4810-937a-da520c439f18} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:45398). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:04.324561 [ 2910 ] {ce628190-fba6-4810-937a-da520c439f18} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:45398). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.328853 [ 1458 ] {e08b52f4-0676-4a7f-933c-9c68f35fdc82} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:53528). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:04.329511 [ 2910 ] {ce628190-fba6-4810-937a-da520c439f18} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:04.329733 [ 1458 ] {e08b52f4-0676-4a7f-933c-9c68f35fdc82} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:53528). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1955: DB::TCPHandler::sendData(DB::Block const&) @ 0x2c7b46b4 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:905: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b242a in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.337958 [ 1458 ] {e08b52f4-0676-4a7f-933c-9c68f35fdc82} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:04.466366 [ 729 ] {85375e91-85f4-4476-9151-9ab95e6b6167} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56230). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:04.466972 [ 729 ] {85375e91-85f4-4476-9151-9ab95e6b6167} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:56230). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.470454 [ 729 ] {85375e91-85f4-4476-9151-9ab95e6b6167} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:04.525506 [ 2918 ] {e5d7efc0-5d45-44c3-9e59-026234e42334} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:34504). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:04.526187 [ 2918 ] {e5d7efc0-5d45-44c3-9e59-026234e42334} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:34504). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:78: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9b1c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:2039: DB::TCPHandler::sendProgress() @ 0x2c7b261d in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:909: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x2c7b249b in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:442: DB::TCPHandler::runImpl() @ 0x2c7a9591 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:04.526761 [ 2918 ] {e5d7efc0-5d45-44c3-9e59-026234e42334} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:07.302915 [ 2914 ] {bbee95c1-f2f2-4b1c-914c-981e17439a92} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008016708 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59264) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.323781 [ 2910 ] {ce85d38d-cb7d-4c54-9ff4-891481913819} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.180032121 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:42556) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.346925 [ 2918 ] {446d13ca-f7da-447e-a788-ede972e10ef2} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.150354712 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36386) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.370568 [ 1458 ] {fdf82a46-651c-4250-8322-1ad05c9ca89f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.13660509 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47286) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.387340 [ 2920 ] {5f6ac69d-40e9-4671-a992-64b1933dc844} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.061779565 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50744) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.389142 [ 729 ] {098899cd-1c89-424f-8717-45929885c308} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.15758234 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34378) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.400661 [ 2916 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:54546). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x2ab9546c in /usr/bin/clickhouse
6. /build/src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x2ab94828 in /usr/bin/clickhouse
7. /build/src/IO/ReadBuffer.h:66: DB::ReadBuffer::next() @ 0x197399fc in /usr/bin/clickhouse
8. /build/src/IO/ReadBuffer.h:100: DB::TCPHandler::runImpl() @ 0x2c7a7fa6 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
10. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
12. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
15. ? @ 0x7f4161c1e609 in ?
16. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:07.402797 [ 1259 ] {9a31472d-b1cf-4ad7-9d8a-8b8b0d8a2476} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.123714237 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42848) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.533554 [ 2918 ] {446d13ca-f7da-447e-a788-ede972e10ef2} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:36386). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:07.533901 [ 1458 ] {fdf82a46-651c-4250-8322-1ad05c9ca89f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.13660509 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.534446 [ 2918 ] {446d13ca-f7da-447e-a788-ede972e10ef2} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.150354712 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.548156 [ 1458 ] {fdf82a46-651c-4250-8322-1ad05c9ca89f} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:07.564764 [ 2918 ] {446d13ca-f7da-447e-a788-ede972e10ef2} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:07.619554 [ 2914 ] {bbee95c1-f2f2-4b1c-914c-981e17439a92} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:59264). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:07.620271 [ 2914 ] {bbee95c1-f2f2-4b1c-914c-981e17439a92} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008016708 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.624168 [ 2914 ] {bbee95c1-f2f2-4b1c-914c-981e17439a92} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:07.633496 [ 729 ] {098899cd-1c89-424f-8717-45929885c308} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:34378). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:07.634248 [ 729 ] {098899cd-1c89-424f-8717-45929885c308} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.15758234 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.650524 [ 729 ] {098899cd-1c89-424f-8717-45929885c308} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:07.657605 [ 1259 ] {9a31472d-b1cf-4ad7-9d8a-8b8b0d8a2476} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.123714237 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.665721 [ 1259 ] {9a31472d-b1cf-4ad7-9d8a-8b8b0d8a2476} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:07.730010 [ 2920 ] {5f6ac69d-40e9-4671-a992-64b1933dc844} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.061779565 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:07.737401 [ 2920 ] {5f6ac69d-40e9-4671-a992-64b1933dc844} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:07.831539 [ 2910 ] {ce85d38d-cb7d-4c54-9ff4-891481913819} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.180032121 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.670872 [ 2920 ] {86efd9b7-99ed-4ac2-8be3-292f92fd69ec} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008285381 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47292) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.708014 [ 2910 ] {223cc428-dc99-4244-aacc-c8c3ea22a81a} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.139950331 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:42572) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.717227 [ 2922 ] {7a6583f9-dc89-4ec5-b331-ffe53eb1f6b2} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.077322949 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36392) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.719432 [ 1458 ] {0955d77f-85c0-4226-9cab-269df7db474f} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.133296075 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50750) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.725126 [ 2918 ] {d6c90a17-093b-408e-b8bf-48995b582929} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.047233166 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42864) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.725272 [ 2914 ] {1cdc5f3d-11c6-4b83-9cc4-dd8c705cfb76} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.077519003 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49688) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.739753 [ 729 ] {a061faa8-2a70-438e-ae43-c25d7d9b6279} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.11549786 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54550) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.744778 [ 1259 ] {3e2ec5a6-5963-4bc0-afb4-43bd2299f9cd} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.085328013 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34394) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.782124 [ 1458 ] {0955d77f-85c0-4226-9cab-269df7db474f} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.133296075 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.799752 [ 729 ] {a061faa8-2a70-438e-ae43-c25d7d9b6279} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.11549786 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.804638 [ 2914 ] {1cdc5f3d-11c6-4b83-9cc4-dd8c705cfb76} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.077519003 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.807724 [ 1458 ] {0955d77f-85c0-4226-9cab-269df7db474f} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:10.808316 [ 1259 ] {3e2ec5a6-5963-4bc0-afb4-43bd2299f9cd} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.085328013 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.811074 [ 729 ] {a061faa8-2a70-438e-ae43-c25d7d9b6279} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:10.811328 [ 2916 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:59274). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x2ab9546c in /usr/bin/clickhouse
6. /build/src/IO/ReadBufferFromPocoSocket.cpp:65: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x2ab94828 in /usr/bin/clickhouse
7. /build/src/IO/ReadBuffer.h:66: DB::ReadBuffer::next() @ 0x197399fc in /usr/bin/clickhouse
8. /build/src/IO/ReadBuffer.h:100: DB::TCPHandler::runImpl() @ 0x2c7a7fa6 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
10. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
11. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
12. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
13. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
15. ? @ 0x7f4161c1e609 in ?
16. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:10.811969 [ 2914 ] {1cdc5f3d-11c6-4b83-9cc4-dd8c705cfb76} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:10.814891 [ 1259 ] {3e2ec5a6-5963-4bc0-afb4-43bd2299f9cd} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:10.819872 [ 2918 ] {d6c90a17-093b-408e-b8bf-48995b582929} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:42864). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:10.820611 [ 2918 ] {d6c90a17-093b-408e-b8bf-48995b582929} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.047233166 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.821978 [ 2918 ] {d6c90a17-093b-408e-b8bf-48995b582929} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:10.847162 [ 2920 ] {86efd9b7-99ed-4ac2-8be3-292f92fd69ec} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.008285381 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.848010 [ 2920 ] {86efd9b7-99ed-4ac2-8be3-292f92fd69ec} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:10.916960 [ 2922 ] {7a6583f9-dc89-4ec5-b331-ffe53eb1f6b2} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.077322949 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:10.917826 [ 2922 ] {7a6583f9-dc89-4ec5-b331-ffe53eb1f6b2} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:11.057524 [ 2910 ] {223cc428-dc99-4244-aacc-c8c3ea22a81a} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.139950331 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.081851 [ 2910 ] {c6eb50b1-7b3e-4f28-a060-c28e87883b96} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.323943658 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::1]:42580) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.278196 [ 1259 ] {0c09348b-7042-4c5a-9072-84b927790d6e} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.438368032 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57252) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.297730 [ 729 ] {bc54bd99-cae5-4d5d-9a2e-89eef78205d7} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.489336757 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:49692) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.298865 [ 2918 ] {14555add-9100-4f02-b493-d18d3b43d658} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.483501467 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59280) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.300887 [ 2920 ] {78ce6fae-abdf-4ed9-8149-a46fafee0f80} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.01995667 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50756) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.300946 [ 2916 ] {8ceeb5d6-aec4-41ef-8199-c408d4410e4a} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.007957196 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:36404) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.308106 [ 2914 ] {bb62d312-5a4d-4ab7-b40d-6dccdf5ef619} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.437828624 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42866) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.308144 [ 1458 ] {4ea01d1a-a6f4-495b-92db-d9f01295ca72} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.474424983 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34406) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.308158 [ 2924 ] {ae83bf5d-c52e-4bb4-a4d9-00a2a0dc9acf} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.434583667 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54554) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.329928 [ 2922 ] {6cf3944d-2ac5-45bd-9124-fa4639464bea} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.501814108 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 23.3.19.34.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:47306) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` GROUP BY `k` ORDER BY `c` DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.484755 [ 2910 ] {c6eb50b1-7b3e-4f28-a060-c28e87883b96} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.323943658 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.554816 [ 2924 ] {ae83bf5d-c52e-4bb4-a4d9-00a2a0dc9acf} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.434583667 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.566627 [ 2924 ] {ae83bf5d-c52e-4bb4-a4d9-00a2a0dc9acf} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:14.572536 [ 2918 ] {14555add-9100-4f02-b493-d18d3b43d658} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:59280). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:14.573868 [ 2918 ] {14555add-9100-4f02-b493-d18d3b43d658} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.483501467 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.575681 [ 2918 ] {14555add-9100-4f02-b493-d18d3b43d658} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:14.622648 [ 1259 ] {0c09348b-7042-4c5a-9072-84b927790d6e} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.438368032 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.633662 [ 1259 ] {0c09348b-7042-4c5a-9072-84b927790d6e} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:14.638275 [ 729 ] {bc54bd99-cae5-4d5d-9a2e-89eef78205d7} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:49692). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:14.639010 [ 729 ] {bc54bd99-cae5-4d5d-9a2e-89eef78205d7} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.489336757 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.644512 [ 2920 ] {78ce6fae-abdf-4ed9-8149-a46fafee0f80} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.01995667 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.649690 [ 2916 ] {8ceeb5d6-aec4-41ef-8199-c408d4410e4a} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.007957196 seconds, maximum: 1: While executing MergeTreeThread. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/QueryPipeline/ReadProgressCallback.cpp:50: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x29901db1 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/ExecutionThreadContext.cpp:59: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) @ 0x2c8516dc in /usr/bin/clickhouse
10. /build/src/Processors/Executors/ExecutionThreadContext.cpp:92: DB::ExecutionThreadContext::executeTask() @ 0x2c851200 in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:246: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x2c830ec1 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PipelineExecutor.cpp:212: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x2c8311d7 in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PipelineExecutor.cpp:337: DB::PipelineExecutor::spawnThreads()::$_0::operator()() const @ 0x2c8328f8 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&) @ 0x2c832855 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c832839 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&>(DB::PipelineExecutor::spawnThreads()::$_0&, std::tuple<>&) @ 0x2c83279d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()::operator()() @ 0x2c8326a2 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(DB::PipelineExecutor::spawnThreads()::$_0&&) @ 0x2c832615 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'()&) @ 0x2c8325f5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c8325d5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PipelineExecutor::spawnThreads()::$_0>(DB::PipelineExecutor::spawnThreads()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c8325a0 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.650928 [ 729 ] {bc54bd99-cae5-4d5d-9a2e-89eef78205d7} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:14.654909 [ 2920 ] {78ce6fae-abdf-4ed9-8149-a46fafee0f80} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:14.680389 [ 2916 ] {8ceeb5d6-aec4-41ef-8199-c408d4410e4a} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:14.729795 [ 2914 ] {bb62d312-5a4d-4ab7-b40d-6dccdf5ef619} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.437828624 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.739305 [ 2914 ] {bb62d312-5a4d-4ab7-b40d-6dccdf5ef619} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:14.739720 [ 2922 ] {6cf3944d-2ac5-45bd-9124-fa4639464bea} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:47306). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:25: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x2ab9b948 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:67: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x2ab9af24 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:49: DB::WriteBuffer::next() @ 0x19725e37 in /usr/bin/clickhouse
8. /build/src/Server/TCPHandler.cpp:1991: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x2c7bb8d5 in /usr/bin/clickhouse
9. /build/src/Server/TCPHandler.cpp:2068: DB::TCPHandler::sendLogs() @ 0x2c7b287f in /usr/bin/clickhouse
10. /build/src/Server/TCPHandler.cpp:598: DB::TCPHandler::runImpl() @ 0x2c7aab48 in /usr/bin/clickhouse
11. /build/src/Server/TCPHandler.cpp:2077: DB::TCPHandler::run() @ 0x2c7bba05 in /usr/bin/clickhouse
12. /build/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x31c260d9 in /usr/bin/clickhouse
13. /build/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x31c2691c in /usr/bin/clickhouse
14. /build/base/poco/Foundation/src/ThreadPool.cpp:188: Poco::PooledThread::run() @ 0x31e79074 in /usr/bin/clickhouse
15. /build/base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x31e75e1a in /usr/bin/clickhouse
16. /build/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x31e74b1e in /usr/bin/clickhouse
17. ? @ 0x7f4161c1e609 in ?
18. __clone @ 0x7f4161b43353 in ?
 (version 23.3.19.34.altinitystable (altinity build))
2024.03.27 10:41:14.740583 [ 2922 ] {6cf3944d-2ac5-45bd-9124-fa4639464bea} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.501814108 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:14.751551 [ 2922 ] {6cf3944d-2ac5-45bd-9124-fa4639464bea} <Warning> TCPHandler: Client has gone away.
2024.03.27 10:41:15.018846 [ 1458 ] {4ea01d1a-a6f4-495b-92db-d9f01295ca72} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.474424983 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x196c4ec2 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x196c4e8d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x31de9f60 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2215df4e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:60: DB::Exception::Exception(String&&, int, bool) @ 0x196b818a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:87: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29904a0c in /usr/bin/clickhouse
6. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:110: bool DB::handleOverflowMode<double, double>(DB::OverflowMode, int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x29903e93 in /usr/bin/clickhouse
7. /build/src/QueryPipeline/ExecutionSpeedLimits.cpp:126: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x29903da8 in /usr/bin/clickhouse
8. /build/src/Interpreters/ProcessList.cpp:458: DB::QueryStatus::checkTimeLimit() @ 0x2af435d9 in /usr/bin/clickhouse
9. /build/src/Processors/Executors/PipelineExecutor.cpp:180: DB::PipelineExecutor::checkTimeLimit() @ 0x2c8300c0 in /usr/bin/clickhouse
10. /build/src/Processors/Executors/PipelineExecutor.cpp:190: DB::PipelineExecutor::finalizeExecution() @ 0x2c83079c in /usr/bin/clickhouse
11. /build/src/Processors/Executors/PipelineExecutor.cpp:124: DB::PipelineExecutor::execute(unsigned long) @ 0x2c830038 in /usr/bin/clickhouse
12. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:83: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x2c8581fa in /usr/bin/clickhouse
13. /build/src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:112: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x2c8580a0 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>()()) std::__invoke[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x2c858035 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x2c858019 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&, std::tuple<>&) @ 0x2c857f7d in /usr/bin/clickhouse
17. /build/src/Common/ThreadPool.h:227: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()::operator()() @ 0x2c857e82 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x2c857df5 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&) @ 0x2c857dd5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x2c857db5 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x2c857d80 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x221b75f6 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x221b6a35 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:396: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x222ad129 in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.cpp:180: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x222b48e4 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x222b4895 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x222b47bd in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x222b42a2 in /usr/bin/clickhouse
29. ? @ 0x7f4161c1e609 in ?
30. __clone @ 0x7f4161b43353 in ?

2024.03.27 10:41:15.026892 [ 1458 ] {4ea01d1a-a6f4-495b-92db-d9f01295ca72} <Warning> TCPHandler: Client has gone away.